-- Logs begin at Wed 2022-06-15 13:46:03 UTC, end at Wed 2022-06-15 15:15:11 UTC. -- Jun 15 14:33:44.906573 dv-cinder-845178-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 14:33:44.913911 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 14:33:44 2022] *** Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: nodename: dv-cinder-845178-7-iscsi Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: machine: x86_64 Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: clock source: unix Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: pcre jit disabled Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: detected number of CPU cores: 4 Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: current working directory: / Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: detected binary path: /usr/bin/uwsgi-core Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: your processes number limit is 47680 Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: your memory page size is 4096 bytes Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: detected max file descriptor number: 2048 Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: lock engine: pthread robust mutexes Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: thunder lock: enabled Jun 15 14:33:44.917448 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 14:33:44.918404 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 14:33:44.961699 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: Python main interpreter initialized at 0x55effb1109c0 Jun 15 14:33:44.961699 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: python threads support enabled Jun 15 14:33:44.961699 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: your server socket listen backlog is limited to 100 connections Jun 15 14:33:44.961699 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: your mercy for graceful operations on workers is 90 seconds Jun 15 14:33:44.962076 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 14:33:44.962153 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: *** Operational MODE: preforking *** Jun 15 14:33:44.962408 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: *** uWSGI is running in multiple interpreter mode *** Jun 15 14:33:44.962408 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: spawned uWSGI master process (pid: 86987) Jun 15 14:33:44.962620 dv-cinder-845178-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 14:33:44.964029 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: spawned uWSGI worker 1 (pid: 87000, cores: 1) Jun 15 14:33:44.964029 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: spawned uWSGI worker 2 (pid: 87001, cores: 1) Jun 15 14:33:44.970234 dv-cinder-845178-7-iscsi devstack@keystone.service[86987]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 14:33:50.532998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:33:50.532998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: from cryptography.utils import int_from_bytes Jun 15 14:33:50.533664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:33:50.533664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: from cryptography.utils import int_from_bytes Jun 15 14:33:50.536478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:33:50.536478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: from cryptography.utils import int_from_bytes Jun 15 14:33:50.536942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 14:33:50.536942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: from cryptography.utils import int_from_bytes Jun 15 14:33:50.583357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:33:50.583553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:33:50.583724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] command line args: [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:33:50.583998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:33:50.584176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ================================================================================ {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:33:50.584431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] admin_token = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.584661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 14:33:50.584740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] config_dir = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.584853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.584930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 14:33:50.585035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] config_source = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.585122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] command line args: [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 14:33:50.585215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] debug = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.585291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 14:33:50.585411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.585672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ================================================================================ {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 14:33:50.585741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] admin_token = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.585806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.585933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] config_dir = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] config_source = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] debug = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] list_limit = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] log_config_append = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.586910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] log_dir = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] log_file = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] list_limit = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.587945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] log_config_append = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] log_dir = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] log_file = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.588965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.589911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.590901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.591076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.591306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.591448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.591523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] publish_errors = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.591599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.591712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.591800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.592460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.592698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.592972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] publish_errors = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] standard_threads = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.593968 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] use_journal = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] use_json = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] use_stderr = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] use_syslog = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.594935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] standard_threads = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.595401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.595579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] use_journal = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.595812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] use_json = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] use_stderr = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.595992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.596069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] use_syslog = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.596214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.596293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 14:33:50.596453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.596528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.596703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.596791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.596940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.597999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.598995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.599145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.599225 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.599397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.599549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.599652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.599795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.599898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.600912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.601982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.602967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.603926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.604996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.605924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.606961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.607108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.607231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.607342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.607418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.608189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.608300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.608524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.608784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.608912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.609041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.609165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.609306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.609429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.609676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.609811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.609998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.610943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611021 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611100 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.611899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.612061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.612265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.612350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.612559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.612709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.612810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.613010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.613094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.613283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.613368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.613520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.613754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.613838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.614003 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.614234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.614346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.614645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.614729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.614900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.615080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.615170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.615370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.615531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.615610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.615833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.615911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.616899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.617055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.617342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.617425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.617600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.617799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.617884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.618867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.619032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.619099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.619264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.619338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.619504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.619571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.619784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.620025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.620290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.620531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.620772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.621964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622390 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.622955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] auth.external = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.623120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.623218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.623561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.623643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.623804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.623871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.624195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.624264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] auth.password = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.624441 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.624630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] auth.token = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.624696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.624939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.625325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.625591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.625835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.626095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.626482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.626552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.626818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.626939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.627072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] auth.external = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.627199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] credential.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.627403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.627561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.627694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.627773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.627941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] auth.password = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] auth.token = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.628983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.629976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] credential.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.630980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.631950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.632933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.633908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.634889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.635997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.636940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637294 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.637982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.638130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.638244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.638402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.638479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.638630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.638720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.638915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.639025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.639292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.639540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.639714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.639804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.639976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.640992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.641981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.642944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.643891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.644992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.645929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.646964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.647917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.648841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.649923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.650898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651293 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.651861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.652959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.653907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.654060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.654138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.654264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.654904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.654999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.655276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.655519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.655705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.655794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.655952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.656925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.657956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] role.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] role.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.658921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.659870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660837 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.660995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.661946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] role.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] role.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.662972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.663945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.664910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.665910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.666053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.666314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.666409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.666552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.666630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.666783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.666860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.667993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.668935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.669967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.670702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.670797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.670957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.671196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.671398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.671491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.671642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.671741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.671885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.671979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.connection = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.672977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.673912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.674888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.675994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.676073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.676229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.676306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.connection = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.676386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87000) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:33:50.676518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.676763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.677000 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.677239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.677472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.677708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.677942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.678208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.678443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.678676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.678918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.679150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.679393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.679626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.679868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.680098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.680336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.680569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 14:33:50.680738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87001) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 14:33:50.793455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.793645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.793825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.796596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.796824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.797056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.803031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.805462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.831291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.831767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.832733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.832963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.884524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.885774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.938842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.939067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.942124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:50.942457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.006683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.006962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.007326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.007514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.007666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.007809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.007809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.008072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.008207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.008348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.008348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.008348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.032155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.032385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.032614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.032706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.032835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.032925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.033898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.044440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.044574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.057500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.057735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.057900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.057996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87000) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.058907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.059033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.059245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87001) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 14:33:51.076493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.076698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:33:51.077043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.077205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.077286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:33:51.077411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:33:51.077681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.077767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.077925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 14:33:51.078011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.078208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.078393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.078480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.078653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.078908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.079162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.089487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.089763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.090028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.090206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.090329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.090466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.090733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.091007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.098614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.099668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.101980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.102288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.102570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.102847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.103126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.103211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.103385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.103466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.103652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.103738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.103920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.104961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.105224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.105489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.105751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.132042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.133651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.133803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.133932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.135504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.135781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.140061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.140340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.141998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.142297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.142574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.147755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.148034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.149902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.150191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.152502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.152785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.153066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.153347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.153629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.153893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.154161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.154502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.155550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.155830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.156105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.156381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.156648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.156915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.157177 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.157439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.173277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.173553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.176435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.176707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.178099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.181244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.181764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.182059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.184869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.185162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.187988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.191283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.191630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.194890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.195414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.195711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.195990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.196267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.196541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.196817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.198823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.199116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.199394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.199671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.199951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.200225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.216419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.216699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.217489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.217799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.220931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.221212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.221486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.221747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.225700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.227443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.227732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.229558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.230799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.231087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.231365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.231638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.232863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.233165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.233469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.235093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.235372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.237015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.237312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.237602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.246759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.247041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.247329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.247607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.247885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.248158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.248984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.251726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.252004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.252282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.252556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.252832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.253104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.253377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.267272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.270550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.272116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.273835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.275372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.275457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.277240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.277517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.277795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.278668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.281341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.283150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.283425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.283701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.285746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.287400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.290656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.290943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.291026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.291224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.291500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.292565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.296449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.296684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.296964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.297234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.301183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.301505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.301788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.306550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.306853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.307129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.309653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87000) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.309933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.310244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.310517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.311322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.311622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.311895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.312168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.312441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.312810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.313084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.313347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.313625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.313894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87000) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.315746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87001) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 14:33:51.316022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.316297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.316569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.316838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.317132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.317400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.317663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.317931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.318329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.318606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.318878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.319152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.319412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87001) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 14:33:51.352489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55effb1109c0 pid: 87000 (default app) Jun 15 14:33:51.355974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc1a9f-7571-417c-94df-598aeadf107a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:33:51.356284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc1a9f-7571-417c-94df-598aeadf107a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:33:51.356565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc1a9f-7571-417c-94df-598aeadf107a None None] PATH_INFO: `/v3` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:33:51.359750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55effb1109c0 pid: 87001 (default app) Jun 15 14:33:51.361317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Wed Jun 15 14:33:51 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:00.185805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec066cb-0b4d-4c7c-996e-11fb21097e25 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:00.186435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec066cb-0b4d-4c7c-996e-11fb21097e25 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:00.186747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec066cb-0b4d-4c7c-996e-11fb21097e25 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:00.191118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:00.196610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:00.197009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:00.197148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:00.267270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: INFO dbcounter [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] Registered counter for database keystone Jun 15 14:34:00.289022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG oslo_db.sqlalchemy.engines [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87000) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:34:00.295939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writer thread running {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:34:00.476214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG passlib.handlers.bcrypt [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87000) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:34:00.476637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG passlib.handlers.bcrypt [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87000) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:34:00.531993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:00.533625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:00.671300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 14:34:00.713704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec066cb-0b4d-4c7c-996e-11fb21097e25 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:00.789559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: INFO dbcounter [None req-cec066cb-0b4d-4c7c-996e-11fb21097e25 None None] Registered counter for database keystone Jun 15 14:34:00.809363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG oslo_db.sqlalchemy.engines [None req-cec066cb-0b4d-4c7c-996e-11fb21097e25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 14:34:00.816277 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writer thread running {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 14:34:00.936271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec066cb-0b4d-4c7c-996e-11fb21097e25 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:00.940570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b620d2-7c41-4ae8-908e-40ac9f0034b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:00.941247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b620d2-7c41-4ae8-908e-40ac9f0034b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:00.941530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b620d2-7c41-4ae8-908e-40ac9f0034b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:00.941807 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b620d2-7c41-4ae8-908e-40ac9f0034b7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:00.950018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b620d2-7c41-4ae8-908e-40ac9f0034b7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:01.010804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING keystone.common.rbac_enforcer.enforcer [None req-74b620d2-7c41-4ae8-908e-40ac9f0034b7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:34:01.011157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b620d2-7c41-4ae8-908e-40ac9f0034b7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:01.022643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Wed Jun 15 14:34:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:34:01.031277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:01.051158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2874b-ab10-4b1e-90ee-6fad62dc2342 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:01.055559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1fbff-44c3-49f3-a370-c3849e6ceecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:01.056711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1fbff-44c3-49f3-a370-c3849e6ceecf admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:01.057001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1fbff-44c3-49f3-a370-c3849e6ceecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:01.057282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1fbff-44c3-49f3-a370-c3849e6ceecf admin admin] PATH_INFO: `/v3/projects/108eb4dc74a441578e74c932122b6873` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:01.058865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1fbff-44c3-49f3-a370-c3849e6ceecf admin admin] RBAC: Authorizing `identity:get_project(project_id=108eb4dc74a441578e74c932122b6873)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:01.111402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING keystone.common.rbac_enforcer.enforcer [None req-68f1fbff-44c3-49f3-a370-c3849e6ceecf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 14:34:01.111751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1fbff-44c3-49f3-a370-c3849e6ceecf admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:01.116547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Wed Jun 15 14:34:01 2022] GET /identity/v3/projects/108eb4dc74a441578e74c932122b6873 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:03.249325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783264d5-f8ae-426c-923f-672adb3b835f None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:03.250075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783264d5-f8ae-426c-923f-672adb3b835f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:03.250075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783264d5-f8ae-426c-923f-672adb3b835f None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:03.254280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:03.259978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e2b3-db09-4fa1-a553-d02b693a0961 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:03.260366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e2b3-db09-4fa1-a553-d02b693a0961 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:03.260536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e2b3-db09-4fa1-a553-d02b693a0961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:03.280839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a8c0e2b3-db09-4fa1-a553-d02b693a0961 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:03.282393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a8c0e2b3-db09-4fa1-a553-d02b693a0961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:03.290182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 14:34:03.313694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783264d5-f8ae-426c-923f-672adb3b835f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:03.326199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783264d5-f8ae-426c-923f-672adb3b835f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:03.331150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba88d4e-187b-4d9b-9100-1f3c10fb7f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:03.331791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba88d4e-187b-4d9b-9100-1f3c10fb7f0b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:03.332076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba88d4e-187b-4d9b-9100-1f3c10fb7f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:03.332610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba88d4e-187b-4d9b-9100-1f3c10fb7f0b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:03.340380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba88d4e-187b-4d9b-9100-1f3c10fb7f0b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:03.342162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba88d4e-187b-4d9b-9100-1f3c10fb7f0b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:03.351020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Wed Jun 15 14:34:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:34:03.356676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e2b3-db09-4fa1-a553-d02b693a0961 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:03.366224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e2b3-db09-4fa1-a553-d02b693a0961 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:03.370559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c170be7-ca44-4a5d-9fe3-9d1fc425d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:03.371513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c170be7-ca44-4a5d-9fe3-9d1fc425d29e admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:03.371877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c170be7-ca44-4a5d-9fe3-9d1fc425d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:03.372243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c170be7-ca44-4a5d-9fe3-9d1fc425d29e admin admin] PATH_INFO: `/v3/users/68464924e2e845d8a0fc9fa4fd5b1824` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:03.374753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c170be7-ca44-4a5d-9fe3-9d1fc425d29e admin admin] RBAC: Authorizing `identity:get_user(user_id=68464924e2e845d8a0fc9fa4fd5b1824)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:03.376674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c170be7-ca44-4a5d-9fe3-9d1fc425d29e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:03.378286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Wed Jun 15 14:34:03 2022] GET /identity/v3/users/68464924e2e845d8a0fc9fa4fd5b1824 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:05.580404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e87161-c9b1-4af2-8e0a-2932afe75172 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:05.580962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e87161-c9b1-4af2-8e0a-2932afe75172 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:05.580962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e87161-c9b1-4af2-8e0a-2932afe75172 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:05.582942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:05.589180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195851-b346-41f3-9fc1-a357cedce8b8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:05.589742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195851-b346-41f3-9fc1-a357cedce8b8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:05.589742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195851-b346-41f3-9fc1-a357cedce8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:05.607751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c6195851-b346-41f3-9fc1-a357cedce8b8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:05.609038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c6195851-b346-41f3-9fc1-a357cedce8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:05.641303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eca0d-23e0-49b9-bd54-51849adfda47 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:05.641582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eca0d-23e0-49b9-bd54-51849adfda47 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:05.642032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eca0d-23e0-49b9-bd54-51849adfda47 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:05.643422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:05.649602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:05.650091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:05.650474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:05.695146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG passlib.handlers.bcrypt [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87001) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 14:34:05.695513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:05.695661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG passlib.handlers.bcrypt [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87001) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 14:34:05.720756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195851-b346-41f3-9fc1-a357cedce8b8 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:05.742002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:05.743672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:05.744721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:05.828483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195851-b346-41f3-9fc1-a357cedce8b8 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:05.849635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:05.862636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:05.863437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:05.863720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:05.863996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:05.872228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:05.875803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:05.887134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:05.888019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:05.888867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:05.896322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:05.981665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:05.988028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb747c6-4cb8-4b71-8533-b1d7bcc05c07 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:05.990523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:06.001786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c9e5-154a-40a7-9ade-79446c0c9240 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:06.029459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:06.030630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:06.030921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:06.031199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:06.039452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:06.040120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:06.040404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:06.040681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:06.041443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:06.041753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:06.043212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:06.056675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:06.063005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:06.111947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:06.113402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:06.185255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ae994-1eea-4311-98bd-ea017d14ab86 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:06.223975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:06.224228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:34:06.224862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:06.225149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:06.225434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] PATH_INFO: `/v3/users/68464924e2e845d8a0fc9fa4fd5b1824` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:06.226707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] RBAC: Authorizing `identity:get_user(user_id=68464924e2e845d8a0fc9fa4fd5b1824)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:06.228193 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:06.230219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:06.230349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:06 2022] GET /identity/v3/users/68464924e2e845d8a0fc9fa4fd5b1824 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:06.232610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:06.235980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:06.251921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:06.345005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce614d-a456-45bb-96a7-16c0c1df5629 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:06.345239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7521a2-36fe-436c-80e5-a483006b9a6f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:06.380202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:06.380718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:06.380948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:06.381236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:06.381517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:06.381635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:06.381800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:06.382081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:06.382751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:06.384231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:06.385935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:06.391842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:06.401572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:06.403092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:06.425904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:06.427655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:06.428815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:06.428815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:06.428815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:06.428815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:06.428815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:06.432731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:06.490378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:06.501030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:06.518119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d704e50-fc15-4ac5-b422-2dedf41b7c29 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:06.546386 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f4fed-729b-4277-bb0f-46cabd7501ba None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:06.558659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e322c28-ed05-49f1-8a65-32120f832aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:06.559332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322c28-ed05-49f1-8a65-32120f832aff None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:06.559621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322c28-ed05-49f1-8a65-32120f832aff None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:06.559897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322c28-ed05-49f1-8a65-32120f832aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:06.560784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e322c28-ed05-49f1-8a65-32120f832aff None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.domain.id=default, user.id=68464924e2e845d8a0fc9fa4fd5b1824) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:06.561179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e322c28-ed05-49f1-8a65-32120f832aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:06.562920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e322c28-ed05-49f1-8a65-32120f832aff None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:06.570094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Wed Jun 15 14:34:06 2022] GET /identity/v3/role_assignments?user.id=68464924e2e845d8a0fc9fa4fd5b1824&scope.domain.id=default&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 233 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:06.582367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaae2a2-77b0-4c43-9adf-1a41ed7babd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:06.583316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaae2a2-77b0-4c43-9adf-1a41ed7babd5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:06.583601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaae2a2-77b0-4c43-9adf-1a41ed7babd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:06.583876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaae2a2-77b0-4c43-9adf-1a41ed7babd5 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:06.584635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaae2a2-77b0-4c43-9adf-1a41ed7babd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:06.584945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaae2a2-77b0-4c43-9adf-1a41ed7babd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:06.586475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaae2a2-77b0-4c43-9adf-1a41ed7babd5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:06.607107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:09.678395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a11ed-3909-4109-91ea-593644b7b428 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:09.679388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a11ed-3909-4109-91ea-593644b7b428 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:09.679886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a11ed-3909-4109-91ea-593644b7b428 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:09.681450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:09.687580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e74e8c-066c-4ef5-a427-c154c34a8977 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:09.688505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e74e8c-066c-4ef5-a427-c154c34a8977 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:09.689001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e74e8c-066c-4ef5-a427-c154c34a8977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:09.711834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b1e74e8c-066c-4ef5-a427-c154c34a8977 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:09.713046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b1e74e8c-066c-4ef5-a427-c154c34a8977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:09.809918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:09.840814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a11ed-3909-4109-91ea-593644b7b428 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:09.880524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98677cf7-9e4f-4f97-8472-0cd704162a20 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:09.881272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98677cf7-9e4f-4f97-8472-0cd704162a20 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:09.882222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98677cf7-9e4f-4f97-8472-0cd704162a20 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:09.884254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:09.892939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74e0b2-3e4b-4046-8ef3-0776c3422a14 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:09.893532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74e0b2-3e4b-4046-8ef3-0776c3422a14 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:09.894027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74e0b2-3e4b-4046-8ef3-0776c3422a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:09.924821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-fa74e0b2-3e4b-4046-8ef3-0776c3422a14 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:09.926680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fa74e0b2-3e4b-4046-8ef3-0776c3422a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:10.009837 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:10.040023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a11ed-3909-4109-91ea-593644b7b428 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:10.069513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:10.109320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74e0b2-3e4b-4046-8ef3-0776c3422a14 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:10.109825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92406bc7-9cf8-47dd-b15c-2136f2cd4a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:10.110542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92406bc7-9cf8-47dd-b15c-2136f2cd4a8b None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.110830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92406bc7-9cf8-47dd-b15c-2136f2cd4a8b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.111112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92406bc7-9cf8-47dd-b15c-2136f2cd4a8b None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.111892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92406bc7-9cf8-47dd-b15c-2136f2cd4a8b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:10.115754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92406bc7-9cf8-47dd-b15c-2136f2cd4a8b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:10.300423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74e0b2-3e4b-4046-8ef3-0776c3422a14 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:10.388196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 13/26] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Jun 15 14:34:09 2022] POST /identity/v3/roles => generated 225 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:10.389123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:10.389957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3195cb-7714-4113-870d-2317f7f6bbd7 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.390436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3195cb-7714-4113-870d-2317f7f6bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.391024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3195cb-7714-4113-870d-2317f7f6bbd7 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.391133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.391420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.391784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.392906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 14/27] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:10.400959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279dc44-d6a1-460c-b285-a40fb3a70644 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.401500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279dc44-d6a1-460c-b285-a40fb3a70644 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.401945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279dc44-d6a1-460c-b285-a40fb3a70644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.406287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:10.408094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:10.430200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:34:10.443103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:10.446795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:10.447582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:34:10.447582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:34:10.447582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:34:10.447582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:10.447582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 14/28] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:10.453070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3279dc44-d6a1-460c-b285-a40fb3a70644 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:10.453395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:10.462198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3279dc44-d6a1-460c-b285-a40fb3a70644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:10.530455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:10.599932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 15/29] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:10.635011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279dc44-d6a1-460c-b285-a40fb3a70644 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:10.637631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed32cc-2d27-4c72-b5dc-389d24eb4898 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:10.690248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:10.691032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.691219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.691505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.692249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:10.692563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:10.693967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:10.707738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 15/30] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:10.713919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:10.827373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:10.848323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379f594-9a4f-43f2-8b25-423d59bdc2fd None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:10.894714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:10.895753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.896240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.896717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.897917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:10.899761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:10.907694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279dc44-d6a1-460c-b285-a40fb3a70644 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:10.955050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492dee3-fc0d-4909-8402-6cff3d4c9556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:10.955050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492dee3-fc0d-4909-8402-6cff3d4c9556 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.955050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492dee3-fc0d-4909-8402-6cff3d4c9556 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.955050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492dee3-fc0d-4909-8402-6cff3d4c9556 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.955560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:10.966255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492dee3-fc0d-4909-8402-6cff3d4c9556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-a492dee3-fc0d-4909-8402-6cff3d4c9556 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:10.975229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:10.976154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:10.976958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 16/31] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:10.977928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48860b19-ab95-46a1-a751-a6b58ebf8003 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.978092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48860b19-ab95-46a1-a751-a6b58ebf8003 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.978403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48860b19-ab95-46a1-a751-a6b58ebf8003 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:10.980077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 17/32] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:10.987016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965e82-32ed-46e7-be64-b088afc6f05d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:10.987311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965e82-32ed-46e7-be64-b088afc6f05d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:10.987604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965e82-32ed-46e7-be64-b088afc6f05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.010018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e3965e82-32ed-46e7-be64-b088afc6f05d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:11.011288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e3965e82-32ed-46e7-be64-b088afc6f05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:11.085976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 18/33] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:11.086874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965e82-32ed-46e7-be64-b088afc6f05d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:11.205323 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965e82-32ed-46e7-be64-b088afc6f05d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:11.235234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:11.247210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0735b37-07e9-40d4-a7f4-e663275ea407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:11.247901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0735b37-07e9-40d4-a7f4-e663275ea407 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.248193 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0735b37-07e9-40d4-a7f4-e663275ea407 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.248475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0735b37-07e9-40d4-a7f4-e663275ea407 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.249389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0735b37-07e9-40d4-a7f4-e663275ea407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:11.249714 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0735b37-07e9-40d4-a7f4-e663275ea407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:11.251356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0735b37-07e9-40d4-a7f4-e663275ea407 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:11.262622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 19/34] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:11.265901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aafc6-14d6-41db-85a5-6fa92c6ef751 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.266507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aafc6-14d6-41db-85a5-6fa92c6ef751 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.267174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aafc6-14d6-41db-85a5-6fa92c6ef751 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.267462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 16/35] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:34:10 2022] POST /identity/v3/projects => generated 301 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:34:11.268926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:11.269604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 20/36] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:11.270763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5aafc6-14d6-41db-85a5-6fa92c6ef751 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:11.420727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5aafc6-14d6-41db-85a5-6fa92c6ef751 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:11.433223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab70081-3fdc-48b3-b7ca-22ae2f86b33c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:11.470383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ba673-d18a-4cf3-bbd6-802881f2def2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:11.471232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba673-d18a-4cf3-bbd6-802881f2def2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.471523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba673-d18a-4cf3-bbd6-802881f2def2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.471808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba673-d18a-4cf3-bbd6-802881f2def2 None admin] PATH_INFO: `/v3/users/68464924e2e845d8a0fc9fa4fd5b1824` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.473214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ba673-d18a-4cf3-bbd6-802881f2def2 None admin] RBAC: Authorizing `identity:get_user(user_id=68464924e2e845d8a0fc9fa4fd5b1824)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:11.474871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ba673-d18a-4cf3-bbd6-802881f2def2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:11.476905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 21/37] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:11 2022] GET /identity/v3/users/68464924e2e845d8a0fc9fa4fd5b1824 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:11.477989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c273-6507-4b4e-9216-5befa66b267e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.478306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c273-6507-4b4e-9216-5befa66b267e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.478589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c273-6507-4b4e-9216-5befa66b267e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.484007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f89bf-79a1-4c0d-962a-b3abfc1d8352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:11.485958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f89bf-79a1-4c0d-962a-b3abfc1d8352 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.488519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f89bf-79a1-4c0d-962a-b3abfc1d8352 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.489081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f89bf-79a1-4c0d-962a-b3abfc1d8352 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.490137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f89bf-79a1-4c0d-962a-b3abfc1d8352 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:11.492070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f89bf-79a1-4c0d-962a-b3abfc1d8352 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:11.500244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-beb8c273-6507-4b4e-9216-5befa66b267e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:11.501944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-beb8c273-6507-4b4e-9216-5befa66b267e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:11.559492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:11.578166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 22/38] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:11.579042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c273-6507-4b4e-9216-5befa66b267e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:11.620529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:11.626798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 17/39] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Jun 15 14:34:11 2022] POST /identity/v3/roles => generated 219 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:11.628357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dad4c-a514-44ba-83d5-223c714f47ee None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.628813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dad4c-a514-44ba-83d5-223c714f47ee None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.629166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dad4c-a514-44ba-83d5-223c714f47ee None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.634665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 18/40] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:11.635878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dad4c-a514-44ba-83d5-223c714f47ee None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:11.698076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c273-6507-4b4e-9216-5befa66b267e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:11.740690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fba6b-9874-4f41-a10a-e31a96e29bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:11.740690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fba6b-9874-4f41-a10a-e31a96e29bc1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.740690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fba6b-9874-4f41-a10a-e31a96e29bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.740690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fba6b-9874-4f41-a10a-e31a96e29bc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.741770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fba6b-9874-4f41-a10a-e31a96e29bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:11.743335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fba6b-9874-4f41-a10a-e31a96e29bc1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:11.745095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 23/41] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:11.746359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94b35f-6726-4eda-9008-ee5fc8eae859 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.746645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94b35f-6726-4eda-9008-ee5fc8eae859 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.746930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94b35f-6726-4eda-9008-ee5fc8eae859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.768264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dad4c-a514-44ba-83d5-223c714f47ee None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:11.771789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1d94b35f-6726-4eda-9008-ee5fc8eae859 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:11.773304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1d94b35f-6726-4eda-9008-ee5fc8eae859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:11.813023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:11.814039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:11.814507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:11.814935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:11.816132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:11.818467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:11.845684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:11.868025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 24/42] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:11.868845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94b35f-6726-4eda-9008-ee5fc8eae859 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:11.910203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 19/43] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Jun 15 14:34:11 2022] POST /identity/v3/roles => generated 223 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:11.910203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:11.986461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94b35f-6726-4eda-9008-ee5fc8eae859 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:11.990540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:12.027952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c082c1-adfc-4d97-84e4-e919cdeb15a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:12.028823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c082c1-adfc-4d97-84e4-e919cdeb15a5 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.029076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c082c1-adfc-4d97-84e4-e919cdeb15a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.029364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c082c1-adfc-4d97-84e4-e919cdeb15a5 None admin] PATH_INFO: `/v3/domains/default/users/68464924e2e845d8a0fc9fa4fd5b1824/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.031792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c082c1-adfc-4d97-84e4-e919cdeb15a5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=68464924e2e845d8a0fc9fa4fd5b1824, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:12.033523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c082c1-adfc-4d97-84e4-e919cdeb15a5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:12.041303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8fcd6-2826-43d8-b27f-eb6b3d1f392e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:12.080737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:12.081549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.081841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.082123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.083759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:12.085294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:12.086976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 20/44] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:12.092529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:12.122652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 25/45] 10.222.0.23 () {58 vars in 1361 bytes} [Wed Jun 15 14:34:11 2022] PUT /identity/v3/domains/default/users/68464924e2e845d8a0fc9fa4fd5b1824/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:12.207835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5093a2-e6d0-47ac-9cb9-ed9f1969b149 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:12.237204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:12.245514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795dbdf-a9e9-4f37-87b7-f4cf7259c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:12.246341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795dbdf-a9e9-4f37-87b7-f4cf7259c560 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:12.246694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795dbdf-a9e9-4f37-87b7-f4cf7259c560 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:12.246895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795dbdf-a9e9-4f37-87b7-f4cf7259c560 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:12.247673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795dbdf-a9e9-4f37-87b7-f4cf7259c560 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:12.249774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795dbdf-a9e9-4f37-87b7-f4cf7259c560 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:12.336585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 21/46] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:34:12 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:34:14.285526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934391f-a18a-476b-bee7-6ab8b9c98060 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.286047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934391f-a18a-476b-bee7-6ab8b9c98060 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.286129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934391f-a18a-476b-bee7-6ab8b9c98060 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.287702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 26/47] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:14.292884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af365e7-45aa-413d-8c1c-e2026b16e2d5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.293349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af365e7-45aa-413d-8c1c-e2026b16e2d5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.293456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af365e7-45aa-413d-8c1c-e2026b16e2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.311297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-0af365e7-45aa-413d-8c1c-e2026b16e2d5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:14.313750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0af365e7-45aa-413d-8c1c-e2026b16e2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:14.382527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 22/48] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:14.405753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934391f-a18a-476b-bee7-6ab8b9c98060 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:14.455903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:14.489640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f506f-c6fa-4788-880c-7f72732e434c None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.489968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f506f-c6fa-4788-880c-7f72732e434c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.490281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f506f-c6fa-4788-880c-7f72732e434c None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.491670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 23/49] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:14.496848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157d382-c111-4de6-8fe8-484a63227506 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.497136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157d382-c111-4de6-8fe8-484a63227506 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.497507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157d382-c111-4de6-8fe8-484a63227506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.515295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7157d382-c111-4de6-8fe8-484a63227506 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:14.516066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934391f-a18a-476b-bee7-6ab8b9c98060 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:14.516532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7157d382-c111-4de6-8fe8-484a63227506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:14.555986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:14.556726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.557017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.557301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.569561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:14.577905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:14.578825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:14.579578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 27/50] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:14.585132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:14.587957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 24/51] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:14.611803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157d382-c111-4de6-8fe8-484a63227506 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:14.653379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:14.687784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca031d3-df3e-4513-9c70-f9267250e0f2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:14.689377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:14.713408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157d382-c111-4de6-8fe8-484a63227506 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:14.723901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:14.724587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.724879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.725161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.725922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:14.726268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:14.727839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:14.736858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 28/52] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:14.742346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:14.749662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:14.750983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.751273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.751550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.752765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:14.754439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:14.756254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 25/53] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:14.761731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:14.842469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e4c8d-b128-4474-acc9-15c08431994f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:14.866254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4731d-97cb-47cb-9a9a-d58342186531 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:14.876661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:14.878411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:14.879481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.879777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.880059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] PATH_INFO: `/v3/users/68464924e2e845d8a0fc9fa4fd5b1824` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.881565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] RBAC: Authorizing `identity:get_user(user_id=68464924e2e845d8a0fc9fa4fd5b1824)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:14.883250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:14.884933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 29/54] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:14 2022] GET /identity/v3/users/68464924e2e845d8a0fc9fa4fd5b1824 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:14.891086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:14.903485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:14.904415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:14.904774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:14.905603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:14.906800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:14.908452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:14.925745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:14.993833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732c8c3-90e3-4630-9b7d-3412ed217a9d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:14.996556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 26/55] 10.222.0.23 () {60 vars in 1140 bytes} [Wed Jun 15 14:34:14 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:34:15.034145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa77f5-ed5c-49a0-a58c-87e2a2d547d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:15.035007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa77f5-ed5c-49a0-a58c-87e2a2d547d2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:15.035382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa77f5-ed5c-49a0-a58c-87e2a2d547d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:15.035580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa77f5-ed5c-49a0-a58c-87e2a2d547d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:15.036869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa77f5-ed5c-49a0-a58c-87e2a2d547d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:15.038441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa77f5-ed5c-49a0-a58c-87e2a2d547d2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:15.040140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 30/56] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:15.046272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:15.163928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd9f5-d60e-4034-b0b0-f9d74cfbd70a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:15.199820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907d341-9705-47a8-aeec-a78325ff31f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:15.200502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907d341-9705-47a8-aeec-a78325ff31f7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:15.200793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907d341-9705-47a8-aeec-a78325ff31f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:15.201079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907d341-9705-47a8-aeec-a78325ff31f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:15.201936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907d341-9705-47a8-aeec-a78325ff31f7 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.domain.id=default, user.id=68464924e2e845d8a0fc9fa4fd5b1824) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:15.202974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907d341-9705-47a8-aeec-a78325ff31f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:15.204606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907d341-9705-47a8-aeec-a78325ff31f7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:15.215436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 27/57] 10.222.0.23 () {56 vars in 1322 bytes} [Wed Jun 15 14:34:15 2022] GET /identity/v3/role_assignments?user.id=68464924e2e845d8a0fc9fa4fd5b1824&scope.domain.id=default&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 530 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:17.199290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc52d3-0cb1-456f-98ce-10532cb13367 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:17.200131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc52d3-0cb1-456f-98ce-10532cb13367 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:17.200329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc52d3-0cb1-456f-98ce-10532cb13367 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:17.202988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 31/58] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:17.208856 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5587-20aa-42fe-9d20-0ae13d5d709c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:17.209287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5587-20aa-42fe-9d20-0ae13d5d709c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:17.209423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5587-20aa-42fe-9d20-0ae13d5d709c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:17.232810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-99fe5587-20aa-42fe-9d20-0ae13d5d709c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:17.234961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-99fe5587-20aa-42fe-9d20-0ae13d5d709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:17.257316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:17.306331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 28/59] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:17.331633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc52d3-0cb1-456f-98ce-10532cb13367 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:17.430499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:17.434241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc52d3-0cb1-456f-98ce-10532cb13367 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:17.467348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138b13f-60cf-47a1-9912-c0bfe8f812a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:17.468742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138b13f-60cf-47a1-9912-c0bfe8f812a2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:17.469031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138b13f-60cf-47a1-9912-c0bfe8f812a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:17.469310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138b13f-60cf-47a1-9912-c0bfe8f812a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:17.470526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138b13f-60cf-47a1-9912-c0bfe8f812a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:17.471980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138b13f-60cf-47a1-9912-c0bfe8f812a2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:17.473601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 32/60] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:17.479211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe5587-20aa-42fe-9d20-0ae13d5d709c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:17.579650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe5587-20aa-42fe-9d20-0ae13d5d709c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:17.614065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258ac40-9b75-4379-b655-57c46b7f48c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:17.614775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258ac40-9b75-4379-b655-57c46b7f48c0 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:17.615065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258ac40-9b75-4379-b655-57c46b7f48c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:17.615345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258ac40-9b75-4379-b655-57c46b7f48c0 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:17.616073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258ac40-9b75-4379-b655-57c46b7f48c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:17.617554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258ac40-9b75-4379-b655-57c46b7f48c0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:17.732279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 29/61] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:17 2022] POST /identity/v3/users => generated 275 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:34:21.172186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a186b-635b-4a5a-84c2-0d4fbe50ff15 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.173136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a186b-635b-4a5a-84c2-0d4fbe50ff15 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.173651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a186b-635b-4a5a-84c2-0d4fbe50ff15 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.175295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 33/62] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:21.193391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc98b4-a509-4fc2-bb97-eca7fb7dbd00 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.194321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc98b4-a509-4fc2-bb97-eca7fb7dbd00 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.194741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc98b4-a509-4fc2-bb97-eca7fb7dbd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.219449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-0ccc98b4-a509-4fc2-bb97-eca7fb7dbd00 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:21.220917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0ccc98b4-a509-4fc2-bb97-eca7fb7dbd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:21.224237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:21.317144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 30/63] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:21.369187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a186b-635b-4a5a-84c2-0d4fbe50ff15 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:21.397473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549db0-2c21-4b5c-ba9f-cb83496dd9ea None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.397982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549db0-2c21-4b5c-ba9f-cb83496dd9ea None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.398468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549db0-2c21-4b5c-ba9f-cb83496dd9ea None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.400165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 31/64] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:21.415757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb821d4-f935-4825-ba7b-246692798ac8 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.416394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb821d4-f935-4825-ba7b-246692798ac8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.416822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb821d4-f935-4825-ba7b-246692798ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.447609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-adb821d4-f935-4825-ba7b-246692798ac8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:21.455163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-adb821d4-f935-4825-ba7b-246692798ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:21.643892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 32/65] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 15 14:34:21.646263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c49f-e29f-4dc5-9b87-db0a1f2d2c51 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.646711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c49f-e29f-4dc5-9b87-db0a1f2d2c51 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.647138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c49f-e29f-4dc5-9b87-db0a1f2d2c51 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.648927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 33/66] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:21.650118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a186b-635b-4a5a-84c2-0d4fbe50ff15 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:21.657506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22477d5f-cba0-465b-afd8-b6cc6e153d46 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.665094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22477d5f-cba0-465b-afd8-b6cc6e153d46 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.665593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22477d5f-cba0-465b-afd8-b6cc6e153d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.687155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-22477d5f-cba0-465b-afd8-b6cc6e153d46 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:21.689053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-22477d5f-cba0-465b-afd8-b6cc6e153d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:21.706539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:21.708065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.709081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.709543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.729118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:21.739488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:21.740372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:21.741108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:21.743851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 34/67] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:21.750229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:21.786768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:21.793894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 34/68] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:21.799404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22477d5f-cba0-465b-afd8-b6cc6e153d46 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:21.807990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:21.894266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928f7d8-8682-4f07-a88e-4c7a5b7586bf None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:21.920366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22477d5f-cba0-465b-afd8-b6cc6e153d46 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:21.932723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd41df9-a2e4-4d1d-a439-8270a1586e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:21.933524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd41df9-a2e4-4d1d-a439-8270a1586e74 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.933808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd41df9-a2e4-4d1d-a439-8270a1586e74 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.934087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd41df9-a2e4-4d1d-a439-8270a1586e74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.945255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd41df9-a2e4-4d1d-a439-8270a1586e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-bcd41df9-a2e4-4d1d-a439-8270a1586e74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:21.953089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:21.953979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:21.954790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 35/69] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:21.957932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd88987-6749-48ef-988f-9b44e162456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:21.958594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88987-6749-48ef-988f-9b44e162456e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.959129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88987-6749-48ef-988f-9b44e162456e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.959413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd88987-6749-48ef-988f-9b44e162456e None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.960185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe4eb7-6e98-4565-967d-870a36629e86 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.960478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd88987-6749-48ef-988f-9b44e162456e None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:21.960638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe4eb7-6e98-4565-967d-870a36629e86 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.960801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd88987-6749-48ef-988f-9b44e162456e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:21.961053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe4eb7-6e98-4565-967d-870a36629e86 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.962205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd88987-6749-48ef-988f-9b44e162456e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:21.962645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 36/70] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:21.967134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe4eb7-6e98-4565-967d-870a36629e86 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:21.975883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 35/71] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:21.977208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3627b-a400-4d1b-aaef-991735203bd7 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:21.977520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3627b-a400-4d1b-aaef-991735203bd7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:21.978446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3627b-a400-4d1b-aaef-991735203bd7 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:21.979311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 36/72] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:21.982649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3627b-a400-4d1b-aaef-991735203bd7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.057783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.072933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.090797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe4eb7-6e98-4565-967d-870a36629e86 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.113013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3627b-a400-4d1b-aaef-991735203bd7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.134076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206deab6-a255-4a42-ab05-9858ac1281ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.134932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deab6-a255-4a42-ab05-9858ac1281ee None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.135215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deab6-a255-4a42-ab05-9858ac1281ee None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.135518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206deab6-a255-4a42-ab05-9858ac1281ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.147065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206deab6-a255-4a42-ab05-9858ac1281ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-206deab6-a255-4a42-ab05-9858ac1281ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:22.155933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:22.156951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:22.157712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 37/73] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:34:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:22.158781 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72824033-389b-4411-a700-63df9dd54187 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.159076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72824033-389b-4411-a700-63df9dd54187 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.159358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72824033-389b-4411-a700-63df9dd54187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.163563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.164243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.164529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.164810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.165577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:22.165889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.167376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.177985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 37/74] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:22.178728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.178848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-72824033-389b-4411-a700-63df9dd54187 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:22.179936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-72824033-389b-4411-a700-63df9dd54187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:22.259677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 38/75] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:22.260292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc573d7-f784-486d-8280-23227c55900c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.260637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc573d7-f784-486d-8280-23227c55900c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.260866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc573d7-f784-486d-8280-23227c55900c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.278782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-2cc573d7-f784-486d-8280-23227c55900c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:22.279996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2cc573d7-f784-486d-8280-23227c55900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:22.324889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d442ce-e928-47e0-a354-2f39f6a46b65 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.349483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.355145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 39/76] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:22.355996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc573d7-f784-486d-8280-23227c55900c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.361642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.367933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.367933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.367933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.368973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.384027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.385593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.392035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 38/77] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:22.392035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.464246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc573d7-f784-486d-8280-23227c55900c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.492089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26200c2e-760d-43c7-aeb1-e59671bdff56 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.502531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.503220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.503511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.503795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.504559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:22.504877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.506329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.514561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 40/78] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:22.515987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.536755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.537645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.537958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.538265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] PATH_INFO: `/v3/users/68464924e2e845d8a0fc9fa4fd5b1824` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.539489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] RBAC: Authorizing `identity:get_user(user_id=68464924e2e845d8a0fc9fa4fd5b1824)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.540966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.542638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/users/68464924e2e845d8a0fc9fa4fd5b1824 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:22.543416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.596545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.600084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.627517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cc48a-c02f-42ed-85fe-a362be150aba None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.652412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119ab17-8a55-4a50-9078-8741d3337d35 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.663768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.665044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.665340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.665620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.675971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:22.683556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:22.684411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 41/80] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:22.685730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.689560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.690798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.691127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.691365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.692584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.694039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.695944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 40/81] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:22.696480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.797732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082c18e-4320-4edc-ad87-e7caa21e139f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.804319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff3be9-cb58-410f-a585-baaff11479ad None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.824134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.826352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:22.834348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.835010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.835301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.835626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.843690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.844568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.844851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.845127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:22.846519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.847983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.849809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 41/82] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:22.850486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:22.903206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:22.904719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:22.955452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57a5bf-d885-4d18-8ade-8cd479a8b6e7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:22.993930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:22.994596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:22.994882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:22.995158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.012182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 42/83] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:23.013057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.046193 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.053135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.067144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.069805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.119400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582ac39-b6d8-41fb-b5f9-0c20320cd0ab None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:23.156699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:23.157525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:23.157947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:23.158328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.159421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:23.159798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.161544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.171438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 43/84] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:23.172232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.179907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 42/85] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:23.185184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.275719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.277866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.287303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffec3b-ffdb-480a-8377-b48befc12eb3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:23.292277 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e61e-6b0a-451b-8703-2add28f305df None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:23.321364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:23.321962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:23.322280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:23.322561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.323288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.324765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.326487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:23.327648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:23.327942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:23.328543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.392605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 44/86] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:23 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:23.393741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.412161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.413824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.517213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.544472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b87c3-46cf-4681-8461-14e1882350f2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:23.545690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 43/87] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:23.547453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.607415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:23.608239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:23.608613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:23.609149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.611722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.616015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.617551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 45/88] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:23.618811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.648799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.684816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9c973-fca1-4455-924a-938b743e12f7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:23.736832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:23.737247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:23.737875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:23.737875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.739132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.740631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.742330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 44/89] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:23.743113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.776885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe924-2c36-4663-9d55-c31ee575ce6e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:23.813970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.834577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:23.835909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:23.836475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:23.836978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.838792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.840620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 46/90] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:23.844022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.875486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078c3c-eed0-4d97-b335-b5eddbe26e4f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:23.916338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:23.917146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:23.917436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:23.917716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:23.918939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:23.920429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:23.922830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 45/91] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:23.923711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:23.955728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:23.977068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a89a0f-47b3-4a3f-aa1d-555fbc46d928 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:24.020063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:24.021103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:24.021636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:24.022229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:24.023417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=dfee2b3a74c5497aa967fe8ef757093d) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:24.023949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:24.025895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:24.033248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 47/92] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/role_assignments?user.id=dfee2b3a74c5497aa967fe8ef757093d&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:24.034672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:24.039760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e730e-f544-4c63-94f5-e2c183cb9054 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:24.065470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:24.079295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:24.079964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:24.083407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:24.083407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:24.157226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:24.158705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:24.171600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10e982-e701-4923-b157-4a6b73c80f8b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:24.240062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:24.241314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:24.241799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:24.242315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:24.243489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=68464924e2e845d8a0fc9fa4fd5b1824) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:24.244045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:24.245859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:24.261160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 48/93] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:24 2022] GET /identity/v3/role_assignments?user.id=68464924e2e845d8a0fc9fa4fd5b1824&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:24.262564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:24.280998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 46/94] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:24.287102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:24.324252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:24.329060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:24.415863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c82e-bfab-4a8d-a020-89d009a9ec9a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:24.463235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:24.464226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:24.464515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:24.464796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] PATH_INFO: `/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:24.466093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] RBAC: Authorizing `identity:get_project(project_id=1c1d40b2806f44c2bdba5c27d5cf40ba)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:24.467641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:24.470083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 47/95] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:24 2022] GET /identity/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:24.476071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:24.481427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab714ea-2e0d-48a4-a6ce-49a6bb41eba0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:24.521170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce32f3-bec5-48f8-98af-dfda170c6eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:24.522231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce32f3-bec5-48f8-98af-dfda170c6eb8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:24.522823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce32f3-bec5-48f8-98af-dfda170c6eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:24.523410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce32f3-bec5-48f8-98af-dfda170c6eb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:24.524553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce32f3-bec5-48f8-98af-dfda170c6eb8 None admin] RBAC: Adding query filter params (role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=dfee2b3a74c5497aa967fe8ef757093d) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:24.525083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce32f3-bec5-48f8-98af-dfda170c6eb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:24.527589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce32f3-bec5-48f8-98af-dfda170c6eb8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:24.537354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 49/96] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:24 2022] GET /identity/v3/role_assignments?user.id=dfee2b3a74c5497aa967fe8ef757093d&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:24.614416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:24.629940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d7156-92e9-4bac-8974-3227a6c9f7ff None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:24.674103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23eac3-4a0f-4b1d-9eba-6e0a822e66ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:24.674720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23eac3-4a0f-4b1d-9eba-6e0a822e66ad None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:24.675003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23eac3-4a0f-4b1d-9eba-6e0a822e66ad None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:24.675279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23eac3-4a0f-4b1d-9eba-6e0a822e66ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:24.676125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23eac3-4a0f-4b1d-9eba-6e0a822e66ad None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=1c1d40b2806f44c2bdba5c27d5cf40ba, user.id=dfee2b3a74c5497aa967fe8ef757093d) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:24.676440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23eac3-4a0f-4b1d-9eba-6e0a822e66ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:24.677908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23eac3-4a0f-4b1d-9eba-6e0a822e66ad None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:24.684280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:24 2022] GET /identity/v3/role_assignments?user.id=dfee2b3a74c5497aa967fe8ef757093d&scope.project.id=1c1d40b2806f44c2bdba5c27d5cf40ba&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:26.286048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f0bb-859e-4b42-8e95-f8b3a18e9afd None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:26.320667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f0bb-859e-4b42-8e95-f8b3a18e9afd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:26.320667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f0bb-859e-4b42-8e95-f8b3a18e9afd None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:26.320667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 50/98] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:26.320932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eed79-dbd7-457b-b3b2-b2dfcc2dea5c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:26.320932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eed79-dbd7-457b-b3b2-b2dfcc2dea5c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:26.320932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eed79-dbd7-457b-b3b2-b2dfcc2dea5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:26.327250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ce3eed79-dbd7-457b-b3b2-b2dfcc2dea5c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:26.329379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ce3eed79-dbd7-457b-b3b2-b2dfcc2dea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:26.407788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 49/99] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:26.430004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f0bb-859e-4b42-8e95-f8b3a18e9afd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:26.552334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:26.586144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f0bb-859e-4b42-8e95-f8b3a18e9afd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:26.588406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39846e4-5a94-4268-a7d7-2c726d4e5fab None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:26.588693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39846e4-5a94-4268-a7d7-2c726d4e5fab None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:26.589057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39846e4-5a94-4268-a7d7-2c726d4e5fab None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:26.590515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 50/100] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:26.600895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963ded3-93cc-499a-af54-f98c12b5b660 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:26.601304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963ded3-93cc-499a-af54-f98c12b5b660 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:26.602144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963ded3-93cc-499a-af54-f98c12b5b660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:26.643585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b963ded3-93cc-499a-af54-f98c12b5b660 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:26.645138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b963ded3-93cc-499a-af54-f98c12b5b660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:26.648062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:26.649975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:26.650443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:26.650879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:26.666496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:26.676310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:26.677199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:26.677974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:26.680904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 51/101] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:26.690912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:26.784522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 51/102] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:26.820533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963ded3-93cc-499a-af54-f98c12b5b660 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:26.842384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:26.883386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b1fb8-99b1-4f55-9f47-9ddedf603c69 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:26.939811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:26.940500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:26.940786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:26.941061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:26.941826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:26.942135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:26.943558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:26.945997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:26.958846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 52/103] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:26.965094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:27.047910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963ded3-93cc-499a-af54-f98c12b5b660 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:27.133255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:27.133978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.134285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.134568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.136040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:27.139325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:27.139325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 52/104] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:27.149405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:27.207246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea222a-200f-43a3-ad99-b192930febc3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:27.288386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:27.289871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.290377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.290819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] PATH_INFO: `/v3/users/12cb75a1f05f46cd9c36652cf8324f1f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.346413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] RBAC: Authorizing `identity:get_user(user_id=12cb75a1f05f46cd9c36652cf8324f1f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:27.347932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:27.349696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 53/105] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:26 2022] GET /identity/v3/users/12cb75a1f05f46cd9c36652cf8324f1f => generated 289 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:27.356927 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:27.367525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:27.424317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ee5de-c5ae-4dc4-afea-e8106ce8698a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:27.524925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a212b-f964-4dda-a166-867692d58e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:27.525538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:27.526801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a212b-f964-4dda-a166-867692d58e80 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.527247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a212b-f964-4dda-a166-867692d58e80 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.527656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a212b-f964-4dda-a166-867692d58e80 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.528515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a212b-f964-4dda-a166-867692d58e80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:27.530127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a212b-f964-4dda-a166-867692d58e80 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:27.590428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e77f-5399-4b4b-8d7c-f6462445c5bf None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:27.616379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 53/106] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:34:27 2022] POST /identity/v3/projects => generated 302 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:27.621308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4b6d-a533-4521-bb04-3ea7f0e4ebde None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.621308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4b6d-a533-4521-bb04-3ea7f0e4ebde None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.621308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4b6d-a533-4521-bb04-3ea7f0e4ebde None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.621308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 54/107] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:27.622889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568396fb-d661-4e54-be41-1c7c3669e6be None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.624645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568396fb-d661-4e54-be41-1c7c3669e6be None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.625179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568396fb-d661-4e54-be41-1c7c3669e6be None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.628195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 55/108] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:27.631451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eee99a-c2fc-41f1-bba1-5784be7623f3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.631969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eee99a-c2fc-41f1-bba1-5784be7623f3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.632425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eee99a-c2fc-41f1-bba1-5784be7623f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.666986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:27.668011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.668960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.673043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.677487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-72eee99a-c2fc-41f1-bba1-5784be7623f3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:27.679612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-72eee99a-c2fc-41f1-bba1-5784be7623f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:27.831950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:27.834633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:27.836232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 56/109] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:27.839623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eecce-9bf8-4e5c-bbc3-269f7d90622d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:27.840113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eecce-9bf8-4e5c-bbc3-269f7d90622d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:27.840566 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1eecce-9bf8-4e5c-bbc3-269f7d90622d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:27.871406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3a1eecce-9bf8-4e5c-bbc3-269f7d90622d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:27.873178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3a1eecce-9bf8-4e5c-bbc3-269f7d90622d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:27.913707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:27.966382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:27.971468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 54/110] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:34:27.972408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:27.973504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 57/111] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:27.980756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eecce-9bf8-4e5c-bbc3-269f7d90622d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.102679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1eecce-9bf8-4e5c-bbc3-269f7d90622d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.107089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85deae-7463-4077-9582-b60c7cfb7401 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.153832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.154667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.154959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.155244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.159622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.160502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.160791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.161072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.162303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:28.163800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:28.170316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 58/112] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:27 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:28.171335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.178036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:28.185953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:28.187502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:28.188237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:28.188797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 55/113] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:28.189374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.234776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:28.248838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:28.295341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a2166-5617-4df9-bb73-58d1c97bf5f6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.316693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716045f-ffe8-4014-8cc1-a09984c18856 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.333003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ca815-a142-49e3-81e0-a367eac4f769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.333671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ca815-a142-49e3-81e0-a367eac4f769 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.333959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ca815-a142-49e3-81e0-a367eac4f769 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.334514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ca815-a142-49e3-81e0-a367eac4f769 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.347279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ca815-a142-49e3-81e0-a367eac4f769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:28.354893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd909b-05d0-421c-b854-4afad2fc93aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.355568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd909b-05d0-421c-b854-4afad2fc93aa None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.355883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd909b-05d0-421c-b854-4afad2fc93aa None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.356167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd909b-05d0-421c-b854-4afad2fc93aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f83ca815-a142-49e3-81e0-a367eac4f769 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:28.356264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:28.357057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:28.357780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:28.358384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd909b-05d0-421c-b854-4afad2fc93aa None admin] RBAC: Adding query filter params (role.id=60619ffdb13e417e9263be4069ea17cd, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=12cb75a1f05f46cd9c36652cf8324f1f) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:28.358384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd909b-05d0-421c-b854-4afad2fc93aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:28.358507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 59/114] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:28.364651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00056457-3ea2-489a-8045-a3b5205fe10c None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.364777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd909b-05d0-421c-b854-4afad2fc93aa None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:28.364858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00056457-3ea2-489a-8045-a3b5205fe10c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.364858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00056457-3ea2-489a-8045-a3b5205fe10c None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.364858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 60/115] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:28.364858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00056457-3ea2-489a-8045-a3b5205fe10c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.378430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 56/116] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/role_assignments?user.id=12cb75a1f05f46cd9c36652cf8324f1f&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=60619ffdb13e417e9263be4069ea17cd => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:28.378430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f09a-69f0-44b9-bc07-4b3a20892d68 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.378430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f09a-69f0-44b9-bc07-4b3a20892d68 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.378430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f09a-69f0-44b9-bc07-4b3a20892d68 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.380988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 57/117] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:28.381542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f09a-69f0-44b9-bc07-4b3a20892d68 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.495721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00056457-3ea2-489a-8045-a3b5205fe10c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.496208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:28.519747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f09a-69f0-44b9-bc07-4b3a20892d68 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.523671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:28.540864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac776c6-03f9-4e7a-b386-55cab3ce7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.541567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac776c6-03f9-4e7a-b386-55cab3ce7aaf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.541853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac776c6-03f9-4e7a-b386-55cab3ce7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.542130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac776c6-03f9-4e7a-b386-55cab3ce7aaf None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.543084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac776c6-03f9-4e7a-b386-55cab3ce7aaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:28.543402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac776c6-03f9-4e7a-b386-55cab3ce7aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:28.544946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac776c6-03f9-4e7a-b386-55cab3ce7aaf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:28.555183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 61/118] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:28.556327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce80a8-9d0e-456e-bb1f-6f2914320afe None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.556670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce80a8-9d0e-456e-bb1f-6f2914320afe None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.556889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce80a8-9d0e-456e-bb1f-6f2914320afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.568387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bd312-40d6-459a-83c6-342f5056f603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.569374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd312-40d6-459a-83c6-342f5056f603 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.569880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd312-40d6-459a-83c6-342f5056f603 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.570391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd312-40d6-459a-83c6-342f5056f603 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.571351 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd312-40d6-459a-83c6-342f5056f603 None admin] RBAC: Adding query filter params (name=member) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:28.571662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd312-40d6-459a-83c6-342f5056f603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:28.574051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd312-40d6-459a-83c6-342f5056f603 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:28.581349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-23ce80a8-9d0e-456e-bb1f-6f2914320afe None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:28.582589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-23ce80a8-9d0e-456e-bb1f-6f2914320afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:28.582783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 58/119] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:28.584018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a58f2b-4c47-4d64-88cc-7115eae97e43 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.584591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a58f2b-4c47-4d64-88cc-7115eae97e43 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.584858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a58f2b-4c47-4d64-88cc-7115eae97e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.607756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-79a58f2b-4c47-4d64-88cc-7115eae97e43 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:28.608989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-79a58f2b-4c47-4d64-88cc-7115eae97e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:28.663817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 62/120] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:28.664675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce80a8-9d0e-456e-bb1f-6f2914320afe None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.704774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 59/121] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:28.706270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a58f2b-4c47-4d64-88cc-7115eae97e43 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.806870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce80a8-9d0e-456e-bb1f-6f2914320afe None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.807956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:28.824365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:28.846605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a58f2b-4c47-4d64-88cc-7115eae97e43 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:28.848285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.850384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.850688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.850969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] PATH_INFO: `/v3/users/68464924e2e845d8a0fc9fa4fd5b1824` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.854941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] RBAC: Authorizing `identity:get_user(user_id=68464924e2e845d8a0fc9fa4fd5b1824)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:28.857193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:28.859787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 63/122] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/users/68464924e2e845d8a0fc9fa4fd5b1824 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:28.861694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.892549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:28.893453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:28.893738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:28.894016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:28.895500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:28.897808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:28.900951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 60/123] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:28.902063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:28.993678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f3aa2-7034-4005-8d88-0df6bf5ff0fb None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.033827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bb17b-b779-44e1-8a7e-5f885e9ae0d0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.042619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.044191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.044753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.045628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.063637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:29.075332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:29.076255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:29.077081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 64/124] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:29.077860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.079945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.081147 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.081710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.082239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.093596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:29.097892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:29.106884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:29.108129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:29.109422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:29.111865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 61/125] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:34:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:29.112882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.142649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:29.222009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9217647-2a9c-4bf1-872b-b3d4273691d0 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.242324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822454f-c436-43bc-9d8f-09c7d2ae8ece None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.258812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.259861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.260220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.260568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.262011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:29.263728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:29.266824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 65/126] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:29 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:29.267720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.287432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.288413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.288795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.289116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.290478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:29.292099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:29.293998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 62/127] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:29 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:29.294986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.366142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:29.395940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b61bb-c726-4961-8e14-0b6c8b076c4e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.400947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc7013-232f-421f-8311-015b2ac9bfbf None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.411914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:29.437971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.438782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.439072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.439351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.440168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:29.440485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:29.442085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:29.442905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.443965 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.444737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.445258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.446245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:29.446748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:29.448904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:29.451303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 66/128] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:29.452490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.464803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 63/129] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:29.465914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.578798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3a3ac-41b0-462d-92d3-d8798f7fff5a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.598522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f749d7-c1d0-4558-b2fd-11b9061a0ec7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.617167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.617979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.618302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.618640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/68464924e2e845d8a0fc9fa4fd5b1824/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.620810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74553fbb1344338e5312c0d2bd5d4e, user_id=68464924e2e845d8a0fc9fa4fd5b1824, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:29.622405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:29.631298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:29.637413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.650891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.650891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.650891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/dfee2b3a74c5497aa967fe8ef757093d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.650891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74553fbb1344338e5312c0d2bd5d4e, user_id=dfee2b3a74c5497aa967fe8ef757093d, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:29.650891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:29.706079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 67/130] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:29 2022] PUT /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/68464924e2e845d8a0fc9fa4fd5b1824/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:29.707325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.746823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 64/131] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:29 2022] PUT /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/dfee2b3a74c5497aa967fe8ef757093d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:29.756188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:29.775447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:29.943886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b5a09-ba6a-45ed-81c1-39c76e7baed2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.959723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20246ca1-14d6-4782-8dfa-00266997e46e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:29.987460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:29.988241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:29.988603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:29.988961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:29.990465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:29.992012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:29.993986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 65/132] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:29 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:30.000115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:30.027984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:30.028891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.029177 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.029459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.034993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:30.036498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:30.038547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 68/133] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:29 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:30.043593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:30.073899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:30.110739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661562e7-f56b-43ea-82b6-945ae7ee5850 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:30.148347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:30.149232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.149527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.149918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.151170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:30.152953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:30.154813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 66/134] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:29 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:30.160767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:30.172683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:30.176173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f8ec8-7118-4e7f-bf05-6b7e407dfd54 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:30.212947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:30.213749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.214035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.214362 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] PATH_INFO: `/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.215585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] RBAC: Authorizing `identity:get_project(project_id=1c1d40b2806f44c2bdba5c27d5cf40ba)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:30.217050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:30.219474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 69/135] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:30 2022] GET /identity/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:30.225082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:30.275242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646bede-0f48-446f-8cb9-f57669de8589 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:30.311484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:30.319149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782410b9-8a18-406f-8a75-63a93f0cbf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:30.320088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782410b9-8a18-406f-8a75-63a93f0cbf93 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.320485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782410b9-8a18-406f-8a75-63a93f0cbf93 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.320780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782410b9-8a18-406f-8a75-63a93f0cbf93 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/dfee2b3a74c5497aa967fe8ef757093d/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.323434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782410b9-8a18-406f-8a75-63a93f0cbf93 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74553fbb1344338e5312c0d2bd5d4e, user_id=dfee2b3a74c5497aa967fe8ef757093d, role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:30.325565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782410b9-8a18-406f-8a75-63a93f0cbf93 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:30.343873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da822e7f-1688-4245-8703-7d08f517e68b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:30.381218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3208a2-766d-491a-94fe-db79476beb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:30.382313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3208a2-766d-491a-94fe-db79476beb3e None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.382619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3208a2-766d-491a-94fe-db79476beb3e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.382901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3208a2-766d-491a-94fe-db79476beb3e None admin] PATH_INFO: `/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba/users/dfee2b3a74c5497aa967fe8ef757093d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.385109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3208a2-766d-491a-94fe-db79476beb3e None admin] RBAC: Authorizing `identity:create_grant(project_id=1c1d40b2806f44c2bdba5c27d5cf40ba, user_id=dfee2b3a74c5497aa967fe8ef757093d, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:30.386703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3208a2-766d-491a-94fe-db79476beb3e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:30.412822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 67/136] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:30 2022] PUT /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/dfee2b3a74c5497aa967fe8ef757093d/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:34:30.472550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 70/137] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:30 2022] PUT /identity/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba/users/dfee2b3a74c5497aa967fe8ef757093d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:30.510685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992c9d-f21b-4baa-aa39-1b4bdeebff54 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.511022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992c9d-f21b-4baa-aa39-1b4bdeebff54 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.511265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992c9d-f21b-4baa-aa39-1b4bdeebff54 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.512658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 68/138] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:30.528593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e66af0-ee09-406a-bbc7-8f3dbc48250e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.528593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e66af0-ee09-406a-bbc7-8f3dbc48250e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.528593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e66af0-ee09-406a-bbc7-8f3dbc48250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.559577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-47e66af0-ee09-406a-bbc7-8f3dbc48250e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:30.560881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-47e66af0-ee09-406a-bbc7-8f3dbc48250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:30.593910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:30.656849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 71/139] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:30.690410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992c9d-f21b-4baa-aa39-1b4bdeebff54 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:30.859775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992c9d-f21b-4baa-aa39-1b4bdeebff54 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:30.914926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5df663-ace5-448a-a0b1-99ebed01d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:30.915754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5df663-ace5-448a-a0b1-99ebed01d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:30.916044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5df663-ace5-448a-a0b1-99ebed01d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:30.916326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5df663-ace5-448a-a0b1-99ebed01d7e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:30.917545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5df663-ace5-448a-a0b1-99ebed01d7e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:30.919098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5df663-ace5-448a-a0b1-99ebed01d7e2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:30.921061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 69/140] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:30.926885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e66af0-ee09-406a-bbc7-8f3dbc48250e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:31.065786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e66af0-ee09-406a-bbc7-8f3dbc48250e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:31.079273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c750f-6f64-4d1d-99c0-3cb580379b5d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:31.079682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c750f-6f64-4d1d-99c0-3cb580379b5d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:31.079970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c750f-6f64-4d1d-99c0-3cb580379b5d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:31.081647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 70/141] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:31.087574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d35c-2dc1-4d3f-a839-f0c616c98588 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:31.087873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d35c-2dc1-4d3f-a839-f0c616c98588 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:31.088155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484d35c-2dc1-4d3f-a839-f0c616c98588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:31.115999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:31.116613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:31.117128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:31.117414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:31.118144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:31.119632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:31.120252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-4484d35c-2dc1-4d3f-a839-f0c616c98588 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:31.121525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4484d35c-2dc1-4d3f-a839-f0c616c98588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:31.212565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:31.251018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 72/142] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:30 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:31.251242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 71/143] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:31.297521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:31.337332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:31.467944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca492b-c33e-4260-b187-c9c1994c30ba None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:31.507352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:31.508574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:31.509072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:31.509539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:31.522888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:31.536671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:31.537570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:31.538358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:31.541178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 73/144] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:31.553089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484d35c-2dc1-4d3f-a839-f0c616c98588 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:31.688857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484d35c-2dc1-4d3f-a839-f0c616c98588 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:31.738928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:31.739561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:31.739847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:31.740126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:31.740904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:31.741215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:31.743240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:31.753011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 72/145] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:31.758716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:31.868251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:31.895276 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bce76-eb4d-4366-a3ad-3daf2b4f3a4a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:31.934598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:31.935829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:31.936304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:31.936766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] PATH_INFO: `/v3/users/12cb75a1f05f46cd9c36652cf8324f1f` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:31.938693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] RBAC: Authorizing `identity:get_user(user_id=12cb75a1f05f46cd9c36652cf8324f1f)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:31.940348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:31.942536 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 74/146] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:31 2022] GET /identity/v3/users/12cb75a1f05f46cd9c36652cf8324f1f => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:31.947577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:32.019847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:32.088839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f4142-dcc4-47fa-aa2b-d9337cdc8b1c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:32.135938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f9c5d-f62a-4cac-a30b-74c36cc2ded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:32.136907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9c5d-f62a-4cac-a30b-74c36cc2ded4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:32.137191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9c5d-f62a-4cac-a30b-74c36cc2ded4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:32.137523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9c5d-f62a-4cac-a30b-74c36cc2ded4 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:32.139064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f9c5d-f62a-4cac-a30b-74c36cc2ded4 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:32.141823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f9c5d-f62a-4cac-a30b-74c36cc2ded4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:32.143707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 73/147] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:31 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:32.149566 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:32.286012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68b9e2-9bf5-4967-bcf6-dabebb9b007f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:32.334797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee08de6-9388-4803-b323-ea377b2483a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:32.336052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee08de6-9388-4803-b323-ea377b2483a4 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:32.336626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee08de6-9388-4803-b323-ea377b2483a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:32.337090 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee08de6-9388-4803-b323-ea377b2483a4 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/12cb75a1f05f46cd9c36652cf8324f1f/roles/60619ffdb13e417e9263be4069ea17cd` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:32.339285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee08de6-9388-4803-b323-ea377b2483a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74553fbb1344338e5312c0d2bd5d4e, user_id=12cb75a1f05f46cd9c36652cf8324f1f, role_id=60619ffdb13e417e9263be4069ea17cd)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:32.340952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee08de6-9388-4803-b323-ea377b2483a4 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:32.421026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 75/148] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:32 2022] PUT /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/users/12cb75a1f05f46cd9c36652cf8324f1f/roles/60619ffdb13e417e9263be4069ea17cd => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:32.990523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636980-9282-46b5-868f-988fa3deb3e2 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:32.991033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636980-9282-46b5-868f-988fa3deb3e2 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:32.991126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636980-9282-46b5-868f-988fa3deb3e2 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:32.992458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 74/149] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:32.999594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1355f7-4d12-4b9a-a3b7-79b88b9749c5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:32.999594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1355f7-4d12-4b9a-a3b7-79b88b9749c5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:32.999594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1355f7-4d12-4b9a-a3b7-79b88b9749c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.007566 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:33.023290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ce1355f7-4d12-4b9a-a3b7-79b88b9749c5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:33.024510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ce1355f7-4d12-4b9a-a3b7-79b88b9749c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:33.116270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 76/150] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:33.157563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636980-9282-46b5-868f-988fa3deb3e2 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:33.285157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636980-9282-46b5-868f-988fa3deb3e2 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:33.300538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:33.328520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:33.329372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.329667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.329942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.344482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:33.357022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:33.358093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:33.359317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 75/151] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:33.366875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1355f7-4d12-4b9a-a3b7-79b88b9749c5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:33.494670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1355f7-4d12-4b9a-a3b7-79b88b9749c5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:33.533120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:33.538771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:33.539697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.540155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.540601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.541532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:33.542015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:33.543678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:33.559116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 77/152] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:33.565050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:33.681986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff686d-1fe7-4d92-96ca-9ed2c642824c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:33.720746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dfcab-a36c-4127-ad5c-ed51a1722cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:33.721962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dfcab-a36c-4127-ad5c-ed51a1722cf0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.722632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dfcab-a36c-4127-ad5c-ed51a1722cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.723090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dfcab-a36c-4127-ad5c-ed51a1722cf0 None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.724580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dfcab-a36c-4127-ad5c-ed51a1722cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:33.726285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dfcab-a36c-4127-ad5c-ed51a1722cf0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:33.728176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 76/153] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:33 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:33.733843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:33.849287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140299d-cb66-4182-86c6-4342d2c01635 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:33.854200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f0f30-823c-43bf-998a-42c0dc5fbb01 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.854701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f0f30-823c-43bf-998a-42c0dc5fbb01 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.855149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f0f30-823c-43bf-998a-42c0dc5fbb01 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.856974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 77/154] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:33.864027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99e90-249a-4343-9eef-1a1749211ea4 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.864588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99e90-249a-4343-9eef-1a1749211ea4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.865047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99e90-249a-4343-9eef-1a1749211ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.891871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7fe99e90-249a-4343-9eef-1a1749211ea4 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:33.894981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7fe99e90-249a-4343-9eef-1a1749211ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:33.946592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:33.947727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:33.948175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:33.948609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:33.991887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:34.068440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 78/155] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:34.070183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb39532-d52c-4788-b3a3-3fd6f4d5079d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.070708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb39532-d52c-4788-b3a3-3fd6f4d5079d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.070810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:34.071631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb39532-d52c-4788-b3a3-3fd6f4d5079d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.073134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 79/156] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:34.078652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f29d6-281a-43ba-87a7-b12c02987d4b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.082283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f29d6-281a-43ba-87a7-b12c02987d4b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.082283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f29d6-281a-43ba-87a7-b12c02987d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.090210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:34.090210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:34.108039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-019f29d6-281a-43ba-87a7-b12c02987d4b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:34.109859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-019f29d6-281a-43ba-87a7-b12c02987d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:34.221631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 78/157] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:34.222666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:34.248528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 80/158] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:34.250510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f29d6-281a-43ba-87a7-b12c02987d4b None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:34.326605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:34.335477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73937c80-caf4-4b43-8ee2-2de7853c6645 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:34.377891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:34.379044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.379468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.379906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.398452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:34.407066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:34.408140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:34.409133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:34.415041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 79/159] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:34.416087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:34.477961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f29d6-281a-43ba-87a7-b12c02987d4b None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:34.478572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:34.570522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:34.571557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.571938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.572222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.574904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:34.576421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:34.579058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 81/160] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:34 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:34.579902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:34.624284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5d467-9868-4dca-9e9d-25332fd96ce7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:34.669849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94804002-865a-49a6-89c2-b18144256bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:34.670621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94804002-865a-49a6-89c2-b18144256bca None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.670905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94804002-865a-49a6-89c2-b18144256bca None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.671603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94804002-865a-49a6-89c2-b18144256bca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.692342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94804002-865a-49a6-89c2-b18144256bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-94804002-865a-49a6-89c2-b18144256bca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:34.702101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:34.703451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:34.704833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 80/161] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:34:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:34.705476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab732e-c1db-426e-9199-518a4b44a7b5 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.705476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab732e-c1db-426e-9199-518a4b44a7b5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.705597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab732e-c1db-426e-9199-518a4b44a7b5 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.707001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 81/162] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:34.707699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab732e-c1db-426e-9199-518a4b44a7b5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:34.736169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:34.821826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab732e-c1db-426e-9199-518a4b44a7b5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:34.859877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25720834-b15d-4a93-8852-177d1ad97550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:34.860538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25720834-b15d-4a93-8852-177d1ad97550 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.860825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25720834-b15d-4a93-8852-177d1ad97550 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.861107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25720834-b15d-4a93-8852-177d1ad97550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.861946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25720834-b15d-4a93-8852-177d1ad97550 None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=dfee2b3a74c5497aa967fe8ef757093d) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:34.862302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25720834-b15d-4a93-8852-177d1ad97550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:34.864126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25720834-b15d-4a93-8852-177d1ad97550 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:34.871035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 82/163] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:34 2022] GET /identity/v3/role_assignments?user.id=dfee2b3a74c5497aa967fe8ef757093d&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:34.872326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db54f-b8be-4e7a-b44a-0de7574ba1e4 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.872614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db54f-b8be-4e7a-b44a-0de7574ba1e4 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.872892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db54f-b8be-4e7a-b44a-0de7574ba1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.876318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e08ad4-4d65-49dc-b699-a9db3bf1c4cf None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:34.901490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b43db54f-b8be-4e7a-b44a-0de7574ba1e4 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:34.902717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b43db54f-b8be-4e7a-b44a-0de7574ba1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:34.956304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:34.956990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:34.957275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:34.957548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:34.958345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:34.958650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:34.960371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:34.978216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 82/164] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:34.979124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:35.028846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 83/165] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:35.029800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db54f-b8be-4e7a-b44a-0de7574ba1e4 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:35.030491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:35.045804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:35.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab55dca-2c34-46cd-ae56-41c7cfc4cfa4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:35.190382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db54f-b8be-4e7a-b44a-0de7574ba1e4 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:35.227889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:35.228499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:35.228786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:35.229065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:35.229830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:35.230142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:35.231595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:35.246230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 83/166] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:35.247105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:35.255222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:35.263977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:35.263977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:35.263977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] PATH_INFO: `/v3/users/68464924e2e845d8a0fc9fa4fd5b1824` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:35.263977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=68464924e2e845d8a0fc9fa4fd5b1824)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:35.263977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:35.266745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 84/167] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:35 2022] GET /identity/v3/users/68464924e2e845d8a0fc9fa4fd5b1824 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:35.267845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:35.383397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:35.393366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c631035-a5bd-4b22-8878-f7c5eb536005 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:35.410008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:35.434705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:35.435849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:35.436500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:35.438987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:35.464806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a1c0f-b3f1-41f7-bc1f-5f2c56c04ec8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:35.470040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:35.487999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:35.488890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:35.489727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:35.493961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 84/168] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:35.497367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:35.545901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:35.547442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:35.555835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:35.556398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:35.558020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:35.560044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:35.562559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 85/169] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:35 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:35.564160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:35.662210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ad18f-ee2d-4538-b2e0-be80a8238c08 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:35.715034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436e03-65dc-451e-a8d6-16b99ebb7aae None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:35.725224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:35.728480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:35.728972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:35.729441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:35.770143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:35.810917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:35.811631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:35.811958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:35.812841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:35.813627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:35.813940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:35.815359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:35.821886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:35.823772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:35.833596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 86/170] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:35.834276 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:35.851250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:35.936385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 85/171] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:35.942598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.003737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.025518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982ade4-9ec5-4cfd-8a3e-1953c608f143 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.055863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bae2d-fcd3-4ed1-b5a5-45af0a529591 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.083805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.084470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.084759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.085293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.097265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.098673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.099162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.099659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] PATH_INFO: `/v3/users/dfee2b3a74c5497aa967fe8ef757093d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.101096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] RBAC: Authorizing `identity:get_user(user_id=dfee2b3a74c5497aa967fe8ef757093d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.102909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.104979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 86/172] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:35 2022] GET /identity/v3/users/dfee2b3a74c5497aa967fe8ef757093d => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:36.106332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.164375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.165860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.173972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.226488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316b7d-bd18-437d-8b99-8f075db872a8 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.253963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.267620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.268814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.269294 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.269879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.270662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 87/173] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:36.271231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.279171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.279171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.279171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 87/174] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:36.280456 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.375214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cb48d-5ce1-4f0e-a19d-2050a83d1e4a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.383055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.399564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342bbd8-7931-4bf0-8ae3-b59605f22193 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.410974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.411628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.411910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.412185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.441835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.443095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.443582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.444126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.445632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.447369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.449551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 88/175] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:36.450812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.483281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.485137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.517752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.579821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece509d2-5831-480b-80cf-ce9813812df2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.590715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 88/176] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:36.591817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.600598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.622490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.623518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.623968 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.624407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.625670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=68464924e2e845d8a0fc9fa4fd5b1824) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:36.626463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.628234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.635328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 89/177] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/role_assignments?user.id=68464924e2e845d8a0fc9fa4fd5b1824&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:36.638793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.720558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9c538-e1d0-4b87-9d3d-6632f5ecfabd None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.764550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d6dae-03d6-4ab5-a62d-b96a3fe18070 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:36.779886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ffff8-3180-4159-a958-3d5af66a91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.780616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ffff8-3180-4159-a958-3d5af66a91b3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.781002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ffff8-3180-4159-a958-3d5af66a91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.781347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ffff8-3180-4159-a958-3d5af66a91b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.782335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ffff8-3180-4159-a958-3d5af66a91b3 None admin] RBAC: Adding query filter params (role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=dfee2b3a74c5497aa967fe8ef757093d) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:36.782654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ffff8-3180-4159-a958-3d5af66a91b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.784245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ffff8-3180-4159-a958-3d5af66a91b3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.794678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.795482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 89/178] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/role_assignments?user.id=dfee2b3a74c5497aa967fe8ef757093d&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:36.797266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb27ef0-447c-4cc7-8151-c9caf577692d None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.797557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb27ef0-447c-4cc7-8151-c9caf577692d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.797839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb27ef0-447c-4cc7-8151-c9caf577692d None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.799309 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 90/179] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:36.805429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c790c3-dbdc-461c-85a3-2c123bafac8c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.805862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c790c3-dbdc-461c-85a3-2c123bafac8c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.806036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c790c3-dbdc-461c-85a3-2c123bafac8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.818324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:36.819861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:36.820372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:36.820886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] PATH_INFO: `/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:36.822617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] RBAC: Authorizing `identity:get_project(project_id=1c1d40b2806f44c2bdba5c27d5cf40ba)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:36.824577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:36.826986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 90/180] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/projects/1c1d40b2806f44c2bdba5c27d5cf40ba => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:36.835256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.839156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-08c790c3-dbdc-461c-85a3-2c123bafac8c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:36.840484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-08c790c3-dbdc-461c-85a3-2c123bafac8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:36.913531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:36.918424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 91/181] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 15 14:34:36.943203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c790c3-dbdc-461c-85a3-2c123bafac8c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:36.976572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b517b3-3a98-486b-a546-4420d629dafa None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.019821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cf1b4-8b03-4a25-b1fb-b766085eafb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.020933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cf1b4-8b03-4a25-b1fb-b766085eafb0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.021424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cf1b4-8b03-4a25-b1fb-b766085eafb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.022039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cf1b4-8b03-4a25-b1fb-b766085eafb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.023278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cf1b4-8b03-4a25-b1fb-b766085eafb0 None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=1c1d40b2806f44c2bdba5c27d5cf40ba, user.id=dfee2b3a74c5497aa967fe8ef757093d) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:37.023822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cf1b4-8b03-4a25-b1fb-b766085eafb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:37.025557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cf1b4-8b03-4a25-b1fb-b766085eafb0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:37.034112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 91/182] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/role_assignments?user.id=dfee2b3a74c5497aa967fe8ef757093d&scope.project.id=1c1d40b2806f44c2bdba5c27d5cf40ba&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:37.036097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049542ba-cb6f-4ed1-bc2c-4d477654b4ba None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.036590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049542ba-cb6f-4ed1-bc2c-4d477654b4ba None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.037167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049542ba-cb6f-4ed1-bc2c-4d477654b4ba None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.038800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 92/183] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:37.048260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed392730-031b-41d4-aecd-3e8aad7f7c8a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.048982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed392730-031b-41d4-aecd-3e8aad7f7c8a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.049496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed392730-031b-41d4-aecd-3e8aad7f7c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.058104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c790c3-dbdc-461c-85a3-2c123bafac8c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.088327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ed392730-031b-41d4-aecd-3e8aad7f7c8a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:37.090341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ed392730-031b-41d4-aecd-3e8aad7f7c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:37.102144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049ec2f-238e-4046-93dd-24472c2f947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.102144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ec2f-238e-4046-93dd-24472c2f947c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.102144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ec2f-238e-4046-93dd-24472c2f947c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.102144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ec2f-238e-4046-93dd-24472c2f947c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.127315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049ec2f-238e-4046-93dd-24472c2f947c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8049ec2f-238e-4046-93dd-24472c2f947c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:37.136862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:37.137919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:37.138740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:37.140727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 92/184] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:37.143346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4666d79-227b-421a-a431-cf95f9fe17b6 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.143770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4666d79-227b-421a-a431-cf95f9fe17b6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.144293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4666d79-227b-421a-a431-cf95f9fe17b6 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.146972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 93/185] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:37.152298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4666d79-227b-421a-a431-cf95f9fe17b6 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.192297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.208647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 93/186] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:37.210859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e953c-c958-4909-9109-a68e2229751f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.215084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e953c-c958-4909-9109-a68e2229751f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.215084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e953c-c958-4909-9109-a68e2229751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.236742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-612e953c-c958-4909-9109-a68e2229751f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:37.238936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.239067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-612e953c-c958-4909-9109-a68e2229751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:37.304826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4666d79-227b-421a-a431-cf95f9fe17b6 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.333912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 94/187] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:37.335403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a2466-a88b-4f72-85ea-9b60c8bbc989 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.335712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a2466-a88b-4f72-85ea-9b60c8bbc989 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.335975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a2466-a88b-4f72-85ea-9b60c8bbc989 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.337274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 95/188] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:37.337978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a2466-a88b-4f72-85ea-9b60c8bbc989 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.349291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1cf24-95c2-4daa-9049-0cf5e503f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.349971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1cf24-95c2-4daa-9049-0cf5e503f948 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.350346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1cf24-95c2-4daa-9049-0cf5e503f948 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.350607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1cf24-95c2-4daa-9049-0cf5e503f948 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.351500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1cf24-95c2-4daa-9049-0cf5e503f948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:37.351746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1cf24-95c2-4daa-9049-0cf5e503f948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:37.353051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1cf24-95c2-4daa-9049-0cf5e503f948 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:37.362932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 94/189] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:37.364449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb081a-53d8-4211-ad84-b238f988328f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.364449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb081a-53d8-4211-ad84-b238f988328f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.364723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb081a-53d8-4211-ad84-b238f988328f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.383276 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-46fb081a-53d8-4211-ad84-b238f988328f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:37.384501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-46fb081a-53d8-4211-ad84-b238f988328f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:37.450216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a2466-a88b-4f72-85ea-9b60c8bbc989 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.462719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 95/190] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:37.463629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb081a-53d8-4211-ad84-b238f988328f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.465182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.485905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.486744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.487023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.487308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.497658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:37.506894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:37.507935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:37.508863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 96/191] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:37.509651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.532699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.580009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb081a-53d8-4211-ad84-b238f988328f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.613813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.614660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.614945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.615224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.616922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed207-98a3-47d2-a9fb-eed2d2583859 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.626017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:37.634185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:37.635212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:37.636122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 96/192] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:37.636923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.655379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.656839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.657114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.657384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] PATH_INFO: `/v3/users/9bd63e581e0e4653a129d2db83456e19` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.670451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] RBAC: Authorizing `identity:get_user(user_id=9bd63e581e0e4653a129d2db83456e19)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:37.672071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:37.673731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 97/193] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/users/9bd63e581e0e4653a129d2db83456e19 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:37.674513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.704929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.738896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dbd59-b547-4d77-9c4c-f62254ae3988 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.751902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.772838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.773631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.773931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.774211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.774757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98677022-5b5f-4f71-b1cf-200aac3fe756 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.775371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:37.776829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:37.778470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 97/194] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:37.779215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.809907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.810607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.810891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.811191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.812471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:37.812797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:37.814208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:37.822982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 98/195] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:37.823766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.898087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a407d-78b1-4a5c-b880-c576080f9a4f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.921702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:37.928312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e06703-e524-4dcf-85b0-311a54e82886 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:37.936025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.936696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.937493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.937821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.938608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:37.938921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:37.940312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:37.948625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 98/196] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:37.949478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:37.964334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:37.965115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:37.965276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:37.965553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:37.992289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.034365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.036187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.049227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8933bc-d774-4466-8f8c-9b16eb6263b9 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.083763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.084433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.084718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.085256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.085971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.087468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.128804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.135903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 99/197] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:38.136729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.196374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 99/198] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:37 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:38.197209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.240441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a4057-c835-4afd-b4d2-3ab374fcd0b1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.280640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.281916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.282231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.282508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] PATH_INFO: `/v3/users/12cb75a1f05f46cd9c36652cf8324f1f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.283779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] RBAC: Authorizing `identity:get_user(user_id=12cb75a1f05f46cd9c36652cf8324f1f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.285269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.287357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 100/199] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/users/12cb75a1f05f46cd9c36652cf8324f1f => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:38.287957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.369670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.379797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.384060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede2c49-6aa6-4d84-bf0b-81964604711b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.412868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d9cd3-02e7-4355-8d0f-8fc9a51bd642 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.427466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.428502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.429282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.429666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] PATH_INFO: `/v3/users/9bd63e581e0e4653a129d2db83456e19` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.431174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] RBAC: Authorizing `identity:get_user(user_id=9bd63e581e0e4653a129d2db83456e19)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.432728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.436843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 100/200] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/users/9bd63e581e0e4653a129d2db83456e19 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:38.438924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.457997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.457997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.457997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.458425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.461397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.464593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.465102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 101/201] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:38.466274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.584830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd613e2b-6be5-41d4-89de-cf2a26d18c3f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.611011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f143-c5b9-4450-85e7-795858cf8ffe None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.623561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.624215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.624537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.624806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.654609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.660437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.661097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.664357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.664357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.666548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.706744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.708449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.750760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.752623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.826327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 101/202] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:38.827179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.878262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 102/203] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:38.879272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:38.907419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.913940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:38.944066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c96c8-cc23-4e79-93f4-391a145ded0d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.982474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:38.983051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:38.983344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:38.984359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:38.985725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=9bd63e581e0e4653a129d2db83456e19) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:38.986040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:38.987650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:38.990918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f783e5-6d8b-4797-b0ff-228e3422652e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:38.994132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 102/204] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/role_assignments?user.id=9bd63e581e0e4653a129d2db83456e19&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:38.995165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:39.048738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:39.049917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:39.051320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:39.051756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:39.058116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:39.059969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:39.062233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 103/205] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:39.072369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:39.208255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b61f90-db76-4b91-bfe5-af1d9a2d876d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:39.238974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:39.244475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70545f-35cb-4b90-950d-63b45dc415b1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:39.248753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:39.259315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:39.260091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:39.260380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:39.260656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:39.261963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:39.263491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:39.265991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 103/206] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:38 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:39.275079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:39.366886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd101d62-60d9-42b3-8ad3-934c9c027348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:39.367691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd101d62-60d9-42b3-8ad3-934c9c027348 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:39.368043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd101d62-60d9-42b3-8ad3-934c9c027348 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:39.368394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd101d62-60d9-42b3-8ad3-934c9c027348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:39.369314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd101d62-60d9-42b3-8ad3-934c9c027348 None admin] RBAC: Adding query filter params (role.id=60619ffdb13e417e9263be4069ea17cd, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e, user.id=12cb75a1f05f46cd9c36652cf8324f1f) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:39.369701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd101d62-60d9-42b3-8ad3-934c9c027348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:39.371263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd101d62-60d9-42b3-8ad3-934c9c027348 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:39.377809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 104/207] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:39 2022] GET /identity/v3/role_assignments?user.id=12cb75a1f05f46cd9c36652cf8324f1f&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=60619ffdb13e417e9263be4069ea17cd => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:39.408668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa49417-78a9-46cd-bce5-6d44a0f72e30 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:39.470230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7821b-2a0d-4c0b-b75a-33602cc64221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:39.470911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7821b-2a0d-4c0b-b75a-33602cc64221 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:39.471197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7821b-2a0d-4c0b-b75a-33602cc64221 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:39.471475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7821b-2a0d-4c0b-b75a-33602cc64221 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:39.472338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7821b-2a0d-4c0b-b75a-33602cc64221 None admin] RBAC: Adding query filter params (role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=9bd63e581e0e4653a129d2db83456e19) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:39.472656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7821b-2a0d-4c0b-b75a-33602cc64221 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:39.474128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7821b-2a0d-4c0b-b75a-33602cc64221 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:39.488014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 104/208] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:39 2022] GET /identity/v3/role_assignments?user.id=9bd63e581e0e4653a129d2db83456e19&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:40.579181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8984-07a2-45cc-a5da-e3e9962715d4 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.580159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8984-07a2-45cc-a5da-e3e9962715d4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.580633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8984-07a2-45cc-a5da-e3e9962715d4 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.582701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 105/209] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:40.588848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec0e1-b082-4e10-ac3d-f4facd52d30b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.589782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec0e1-b082-4e10-ac3d-f4facd52d30b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.590316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec0e1-b082-4e10-ac3d-f4facd52d30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.607580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-957ec0e1-b082-4e10-ac3d-f4facd52d30b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:40.609545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-957ec0e1-b082-4e10-ac3d-f4facd52d30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:40.630098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66642a-12b8-4e6c-8059-7e334896ba8c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.630701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66642a-12b8-4e6c-8059-7e334896ba8c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.631261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66642a-12b8-4e6c-8059-7e334896ba8c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.633174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 106/210] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:40.638865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973157-21ab-4b48-a1d8-f06a9efb9558 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.639154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973157-21ab-4b48-a1d8-f06a9efb9558 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.639430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d973157-21ab-4b48-a1d8-f06a9efb9558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.660820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-5d973157-21ab-4b48-a1d8-f06a9efb9558 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:40.662972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5d973157-21ab-4b48-a1d8-f06a9efb9558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:40.675832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:40.686673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 105/211] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:40.710431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec0e1-b082-4e10-ac3d-f4facd52d30b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:40.756527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 107/212] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:40.776332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973157-21ab-4b48-a1d8-f06a9efb9558 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:40.846387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec0e1-b082-4e10-ac3d-f4facd52d30b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:40.893257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:40.894718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.895409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.895883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.896702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:40.906528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d973157-21ab-4b48-a1d8-f06a9efb9558 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:40.912458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:40.921328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:40.922891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:40.923665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 106/213] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:40.934257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:40.954759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:40.955566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:40.955833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:40.956101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:40.958004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:40.962357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:40.964342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 108/214] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:40.971341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.002460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.062063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbceda1-b9fb-4b50-aeb2-02897f144011 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.094615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6cdb1-271c-493b-a96b-9e1a034a4bd9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.101351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.102071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.102477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.102848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.103815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:41.104204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.105719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.120819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 107/215] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:41.122030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.135849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.136777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.137239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.137686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.138609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.140818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.228592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 109/216] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:40 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:41.253029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c7942-0ff4-4d8b-a91f-c63cd29e3681 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.272760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.298656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de372d8d-f5e2-4fe6-9279-6d0d60bf204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.299628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372d8d-f5e2-4fe6-9279-6d0d60bf204b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.299912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372d8d-f5e2-4fe6-9279-6d0d60bf204b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.300182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372d8d-f5e2-4fe6-9279-6d0d60bf204b None admin] PATH_INFO: `/v3/users/f459dd0493ed420090bc0c7698a35da6` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.321539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de372d8d-f5e2-4fe6-9279-6d0d60bf204b None admin] RBAC: Authorizing `identity:get_user(user_id=f459dd0493ed420090bc0c7698a35da6)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.323221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de372d8d-f5e2-4fe6-9279-6d0d60bf204b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.325417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 108/217] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/users/f459dd0493ed420090bc0c7698a35da6 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:41.332666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.356307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.392645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb531-bfd3-4b41-8a81-1a8d1f2caafa None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.392968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb531-bfd3-4b41-8a81-1a8d1f2caafa None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.393204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb531-bfd3-4b41-8a81-1a8d1f2caafa None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.394681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 109/218] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:41.400462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ecf3-135e-4a0b-b65e-bad831304caf None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.401054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ecf3-135e-4a0b-b65e-bad831304caf None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.401529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ecf3-135e-4a0b-b65e-bad831304caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.419250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-f113ecf3-135e-4a0b-b65e-bad831304caf None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:41.420764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f113ecf3-135e-4a0b-b65e-bad831304caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:41.466661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e150cf1-0f33-455c-8956-fdb4eadc6a1d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.507079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.507784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.508072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.508342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.514213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 110/219] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:41.537829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113ecf3-135e-4a0b-b65e-bad831304caf None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.610575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.611899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.638045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.686263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113ecf3-135e-4a0b-b65e-bad831304caf None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.694195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.742353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.743158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.743444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.743724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.746267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 110/220] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:41.748750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.759800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:41.768593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:41.769514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:41.770324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:41.772772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 111/221] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:41.778540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.890051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817f06b-e78d-45d9-8d43-d50def0dd489 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.905018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd06a2b-88fb-4d18-8ef2-0b91f10d569b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:41.912401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:41.940900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d244234-2d65-4a82-b95c-2b0ed62b26fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.941983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244234-2d65-4a82-b95c-2b0ed62b26fd None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.942296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244234-2d65-4a82-b95c-2b0ed62b26fd None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.942575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244234-2d65-4a82-b95c-2b0ed62b26fd None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.943224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68de9c-de46-4d9d-a13a-10a14f67e36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:41.943784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d244234-2d65-4a82-b95c-2b0ed62b26fd None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.944086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68de9c-de46-4d9d-a13a-10a14f67e36e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.944460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68de9c-de46-4d9d-a13a-10a14f67e36e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.944828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68de9c-de46-4d9d-a13a-10a14f67e36e None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.945267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d244234-2d65-4a82-b95c-2b0ed62b26fd None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.945776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68de9c-de46-4d9d-a13a-10a14f67e36e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:41.946195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68de9c-de46-4d9d-a13a-10a14f67e36e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:41.947213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 111/222] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:41.947788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68de9c-de46-4d9d-a13a-10a14f67e36e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:41.948188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8e83b-beec-43aa-9f8c-6e42ccafd139 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.948473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8e83b-beec-43aa-9f8c-6e42ccafd139 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.948851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8e83b-beec-43aa-9f8c-6e42ccafd139 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.951129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 112/223] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:41.955764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8e83b-beec-43aa-9f8c-6e42ccafd139 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:41.963192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 112/224] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:41.969080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270072bf-a612-46d3-bc06-f48699f68a86 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:41.969741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270072bf-a612-46d3-bc06-f48699f68a86 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:41.970251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270072bf-a612-46d3-bc06-f48699f68a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:41.987942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-270072bf-a612-46d3-bc06-f48699f68a86 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:41.989438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-270072bf-a612-46d3-bc06-f48699f68a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:42.008275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:42.061688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 113/225] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:42.062980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270072bf-a612-46d3-bc06-f48699f68a86 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:42.091729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8e83b-beec-43aa-9f8c-6e42ccafd139 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:42.136120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:42.137034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:42.137325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:42.138005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:42.139166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=f459dd0493ed420090bc0c7698a35da6) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:42.139484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:42.142314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:42.149713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 113/226] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:41 2022] GET /identity/v3/role_assignments?user.id=f459dd0493ed420090bc0c7698a35da6&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:42.150747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:42.217594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270072bf-a612-46d3-bc06-f48699f68a86 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:42.261921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:42.262861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:42.263147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:42.263511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] PATH_INFO: `/v3/users/9bd63e581e0e4653a129d2db83456e19` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:42.265138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=9bd63e581e0e4653a129d2db83456e19)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:42.266802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:42.269278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 114/227] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:42 2022] GET /identity/v3/users/9bd63e581e0e4653a129d2db83456e19 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:42.275097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:42.276791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:42.309192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:42.414138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae7b05-da9e-4781-87a6-c4df8d26b560 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:42.427449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bded3b-02af-4a16-8f12-ee230217c3b6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:42.484783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:42.485645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:42.485932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:42.486223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:42.496759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:42.497631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:42.497911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:42.498280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:42.500176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:42.501657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:42.503512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 115/228] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:42 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:42.509512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:42.514872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:42.523817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:42.524660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:42.525436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 114/229] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:34:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:42.539287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:42.690526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53eabc-ccd9-4bc0-b049-b3835d9b0775 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:42.692321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:42.702037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:42.710845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bb387-aca5-4a46-896f-db57f775efc8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:42.732975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:42.733910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:42.744187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:42.744187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:42.744187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:42.744187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:42.744187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:42.756047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 115/230] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:42.762512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:42.782264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:42.784244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:42.784244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:42.784244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/9bd63e581e0e4653a129d2db83456e19/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:42.789601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=14d7888970604c63a01c4de4b27aa60c, user_id=9bd63e581e0e4653a129d2db83456e19, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:42.791104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:42.874431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0986-3e2e-4b96-8326-9c5bc1cd0e76 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:42.900869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 116/231] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:42 2022] PUT /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/9bd63e581e0e4653a129d2db83456e19/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:42.922696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:42.923906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:42.924341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:42.924756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] PATH_INFO: `/v3/users/9bd63e581e0e4653a129d2db83456e19` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:42.930759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] RBAC: Authorizing `identity:get_user(user_id=9bd63e581e0e4653a129d2db83456e19)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:42.932674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:42.935113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 116/232] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:42 2022] GET /identity/v3/users/9bd63e581e0e4653a129d2db83456e19 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:42.941334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:43.114262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bcc20-947b-464f-b591-bacb29299cb5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:43.164180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bbf39-4e8d-4417-b17e-1383dab541dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:43.165052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bbf39-4e8d-4417-b17e-1383dab541dd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.165332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bbf39-4e8d-4417-b17e-1383dab541dd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.165605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bbf39-4e8d-4417-b17e-1383dab541dd None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.166838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bbf39-4e8d-4417-b17e-1383dab541dd None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:43.168290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bbf39-4e8d-4417-b17e-1383dab541dd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:43.170408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 117/233] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:42 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:43.179237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:43.229439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:43.308381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8ad98-7010-40a7-a71e-754c6c19ef4a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:43.360760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656eca91-6000-4e95-9cf1-11a42c6dd0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:43.361992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eca91-6000-4e95-9cf1-11a42c6dd0e3 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.362465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eca91-6000-4e95-9cf1-11a42c6dd0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.362911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eca91-6000-4e95-9cf1-11a42c6dd0e3 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/9bd63e581e0e4653a129d2db83456e19/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.368021 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656eca91-6000-4e95-9cf1-11a42c6dd0e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=14d7888970604c63a01c4de4b27aa60c, user_id=9bd63e581e0e4653a129d2db83456e19, role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:43.369708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656eca91-6000-4e95-9cf1-11a42c6dd0e3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:43.479290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 117/234] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:43 2022] PUT /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/9bd63e581e0e4653a129d2db83456e19/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:43.784855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf02d-9921-4107-bf97-ff434cac444f None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.785347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf02d-9921-4107-bf97-ff434cac444f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.785347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf02d-9921-4107-bf97-ff434cac444f None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.786774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 118/235] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:43.792387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40d55-f778-4b2d-9a67-5796f0104b8d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:43.793255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40d55-f778-4b2d-9a67-5796f0104b8d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:43.793729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40d55-f778-4b2d-9a67-5796f0104b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:43.814955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f0b40d55-f778-4b2d-9a67-5796f0104b8d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:43.816161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f0b40d55-f778-4b2d-9a67-5796f0104b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:43.895435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 118/236] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:43.920821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bf02d-9921-4107-bf97-ff434cac444f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:43.946321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.088002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7690cb-6f59-4365-8f73-a34ba138d787 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.088660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7690cb-6f59-4365-8f73-a34ba138d787 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.089142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7690cb-6f59-4365-8f73-a34ba138d787 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.090969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 119/237] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:44.094075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bf02d-9921-4107-bf97-ff434cac444f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.097003 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b0bd-1dcf-452e-89c9-94d8c7dc163a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.097516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b0bd-1dcf-452e-89c9-94d8c7dc163a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.098086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b0bd-1dcf-452e-89c9-94d8c7dc163a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.119655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1ec9b0bd-1dcf-452e-89c9-94d8c7dc163a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:44.120831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1ec9b0bd-1dcf-452e-89c9-94d8c7dc163a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:44.150998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.152994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.154289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.154767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.170156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.174406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:44.183769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:44.187573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:44.188339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 119/238] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:44.193565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:44.259340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 120/239] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:44.284461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9b0bd-1dcf-452e-89c9-94d8c7dc163a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:44.337060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.364332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45c50c-443a-497f-8867-0712e3f06c29 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.430711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.431632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.431914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.432187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.432943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:44.433251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:44.434673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:44.450162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 120/240] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:44.457760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:44.467392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9b0bd-1dcf-452e-89c9-94d8c7dc163a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.533811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.534662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.534946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.535223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.536416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:44.539407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:44.541059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 121/241] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:44.546592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:44.594562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.631683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a94bb-4ff2-4d5a-b393-a7ccca615d50 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.705128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cdbb8-cb5f-4dc7-9f52-f26e9e1dc5b0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.715779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.716667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.716954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.717228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] PATH_INFO: `/v3/users/e787c2fb2bc0489896fc066fc1a4591d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.742523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:44.743949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9313f-ad25-461b-b754-6a351c3216e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.744847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9313f-ad25-461b-b754-6a351c3216e7 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.745349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9313f-ad25-461b-b754-6a351c3216e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.745841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9313f-ad25-461b-b754-6a351c3216e7 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.746752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9313f-ad25-461b-b754-6a351c3216e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:44.748441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9313f-ad25-461b-b754-6a351c3216e7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:44.755730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] RBAC: Authorizing `identity:get_user(user_id=e787c2fb2bc0489896fc066fc1a4591d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:44.757341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:44.759413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 121/242] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:44 2022] GET /identity/v3/users/e787c2fb2bc0489896fc066fc1a4591d => generated 297 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:44.766270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:44.826652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 122/243] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:44 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:44.898596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790ead0-3d7f-4b59-be84-f547ab0073f5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:44.949771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057a4cc-a886-4ec2-b4bc-b2a796ee2931 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.949959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:44.950076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057a4cc-a886-4ec2-b4bc-b2a796ee2931 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.950382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057a4cc-a886-4ec2-b4bc-b2a796ee2931 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.951212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.951713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.952166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 123/244] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:44.952366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.960771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9d299-822a-4a84-ae45-1f0c3efe233f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:44.960771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9d299-822a-4a84-ae45-1f0c3efe233f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:44.960771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9d299-822a-4a84-ae45-1f0c3efe233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:44.980737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-09b9d299-822a-4a84-ae45-1f0c3efe233f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:44.982338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-09b9d299-822a-4a84-ae45-1f0c3efe233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:45.070909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:45.077671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:45.080472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:45.097161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 124/245] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:45.119689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9d299-822a-4a84-ae45-1f0c3efe233f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:45.146096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:45.222286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 122/246] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:45.223774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:45.277956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9d299-822a-4a84-ae45-1f0c3efe233f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:45.322687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:45.323551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.323833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.324108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:45.352063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:45.352948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:45.353730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 125/247] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:45.361371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:45.367952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:45.372729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6b3aa-7e49-4677-92af-3413243d8e1e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:45.419417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:45.421234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.421906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.422387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.424019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:45.426084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:45.428662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 123/248] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:45.434815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:45.447761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:45.479744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff58e4-c363-4a06-8448-5db3fde1ea6f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:45.518085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:45.518678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.518963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.519240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.519999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] RBAC: Adding query filter params (name=member) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:45.520308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:45.521736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:45.532173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 126/249] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:45.537077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:45.554137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc1e40-1ad5-4420-ac1e-9d3ac848f8ce None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:45.595504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae684eb6-16f5-4d4e-87ce-d6d69ef6bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:45.596176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684eb6-16f5-4d4e-87ce-d6d69ef6bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.596457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684eb6-16f5-4d4e-87ce-d6d69ef6bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.596732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684eb6-16f5-4d4e-87ce-d6d69ef6bbd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.597670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae684eb6-16f5-4d4e-87ce-d6d69ef6bbd8 None admin] RBAC: Adding query filter params (role.id=60619ffdb13e417e9263be4069ea17cd, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=e787c2fb2bc0489896fc066fc1a4591d) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:45.597986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae684eb6-16f5-4d4e-87ce-d6d69ef6bbd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:45.599735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae684eb6-16f5-4d4e-87ce-d6d69ef6bbd8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:45.607291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 124/250] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/role_assignments?user.id=e787c2fb2bc0489896fc066fc1a4591d&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=60619ffdb13e417e9263be4069ea17cd => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:45.633401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2229cfb-ec59-4683-8894-34f0f3ed4bde None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.633691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2229cfb-ec59-4683-8894-34f0f3ed4bde None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.633974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2229cfb-ec59-4683-8894-34f0f3ed4bde None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.635631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 125/251] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:45.641823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b1490-1215-4485-9fcc-a659e0626f2b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.642139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b1490-1215-4485-9fcc-a659e0626f2b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.642415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b1490-1215-4485-9fcc-a659e0626f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.657371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e7ee7-1e07-4009-8254-62521f9a9d06 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:45.678102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-0c1b1490-1215-4485-9fcc-a659e0626f2b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:45.679672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0c1b1490-1215-4485-9fcc-a659e0626f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:45.706694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:45.707593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:45.707879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:45.708153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] PATH_INFO: `/v3/users/f459dd0493ed420090bc0c7698a35da6` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:45.713073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] RBAC: Authorizing `identity:get_user(user_id=f459dd0493ed420090bc0c7698a35da6)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:45.714611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:45.723073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 127/252] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/users/f459dd0493ed420090bc0c7698a35da6 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:45.732323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:45.745558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:45.750632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:45.831894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 126/253] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:45.871708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b1490-1215-4485-9fcc-a659e0626f2b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:45.918330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5d17d-749e-4ced-b20e-3a7f09554f61 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.000991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.001855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.002136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.002440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.008025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.009495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.011260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 128/254] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:46.019076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.072691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b1490-1215-4485-9fcc-a659e0626f2b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.145811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.146661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.146942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.147210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.167272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:46.175620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:46.176456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:34:46.177213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:46.184284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 127/255] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:34:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:46.190651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.205851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.246580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.264493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f70c0-972d-47cd-952f-af8f9b7636c7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.357241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.358152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.358457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.358731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/f459dd0493ed420090bc0c7698a35da6/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.367939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] RBAC: Authorizing `identity:create_grant(project_id=14d7888970604c63a01c4de4b27aa60c, user_id=f459dd0493ed420090bc0c7698a35da6, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.369437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.420535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bb833-618f-4ec2-8b05-3b71cad0331c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.461327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 129/256] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:46 2022] PUT /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/f459dd0493ed420090bc0c7698a35da6/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:46.474127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60234ef-870e-41d9-be8d-9dd9c8c16629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.475309 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60234ef-870e-41d9-be8d-9dd9c8c16629 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.475832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60234ef-870e-41d9-be8d-9dd9c8c16629 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.476301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60234ef-870e-41d9-be8d-9dd9c8c16629 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.477606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60234ef-870e-41d9-be8d-9dd9c8c16629 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:46.478124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60234ef-870e-41d9-be8d-9dd9c8c16629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.479825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60234ef-870e-41d9-be8d-9dd9c8c16629 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.494754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 128/257] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:46.501436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.525216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed1ed9-e0d3-421a-93d9-75ea1041ef55 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.525811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed1ed9-e0d3-421a-93d9-75ea1041ef55 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.526331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed1ed9-e0d3-421a-93d9-75ea1041ef55 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.528055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 129/258] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:46.537290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5567c-342a-4d35-ab04-a5d2de250a08 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.537647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5567c-342a-4d35-ab04-a5d2de250a08 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.537992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5567c-342a-4d35-ab04-a5d2de250a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.575320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-42d5567c-342a-4d35-ab04-a5d2de250a08 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:46.576542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-42d5567c-342a-4d35-ab04-a5d2de250a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:46.720305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 130/259] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:46.748552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb418e-1660-46a9-a214-8671374c5224 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:46.770694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5567c-342a-4d35-ab04-a5d2de250a08 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.794658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.796194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:46.800347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:46.802284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:46.802666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:46.802950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] PATH_INFO: `/v3/users/9bd63e581e0e4653a129d2db83456e19` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:46.804780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] RBAC: Authorizing `identity:get_user(user_id=9bd63e581e0e4653a129d2db83456e19)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:46.810808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:46.810808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 130/260] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/users/9bd63e581e0e4653a129d2db83456e19 => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:46.816970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:46.967572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5567c-342a-4d35-ab04-a5d2de250a08 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.008277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad7aa3-9fd2-4bbf-bbfb-e712b03c5db4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.034560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.035386 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.035755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.036038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.044255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.044965 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.045193 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.045471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.055518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:47.070208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:47.071209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:47.071966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:47.075040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 131/261] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:47.081204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.130359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.132022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.142212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.224679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.264701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19fd99-ed45-40b2-bf68-5d9292954696 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.304121 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.305141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.305620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.306092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.307099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:47.307604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.309247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.318693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 132/262] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:47.324637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 131/263] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:34:47.325942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.343234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.436650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a292505-64f2-4a1f-b879-133658379b51 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.474205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.475437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.475939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.476408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] PATH_INFO: `/v3/users/9bd63e581e0e4653a129d2db83456e19` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.477821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] RBAC: Authorizing `identity:get_user(user_id=9bd63e581e0e4653a129d2db83456e19)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.479528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.481801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 133/264] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/users/9bd63e581e0e4653a129d2db83456e19 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:47.491156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.494779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.559679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:47.564681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a810d3-734f-4824-8f81-a9cc3ddb9107 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.604077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.605327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.605810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.606315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.607702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.609427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.611815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 132/265] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:47.617642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:47.767703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d871bd6-17b8-483f-ad48-9d44145fb4ad None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.789769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4ac0-c806-46e6-aecd-6454cd361bbf None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:47.831149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.832214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.832694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.833162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.834222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=9bd63e581e0e4653a129d2db83456e19) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:47.834735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.837370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.844321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 133/266] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/role_assignments?user.id=9bd63e581e0e4653a129d2db83456e19&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:47.847787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:47.847787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:47.847787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:47.847787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:47.963813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:47.965650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:47.974540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:48.094923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 134/267] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:34:48.100376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:48.139920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:48.254765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e3ad-b131-4082-863e-9ebdf9a932f7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:48.296592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724acc0-bd57-4123-a6b0-27063a7fce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:48.298140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724acc0-bd57-4123-a6b0-27063a7fce62 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.298445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724acc0-bd57-4123-a6b0-27063a7fce62 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.298718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724acc0-bd57-4123-a6b0-27063a7fce62 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.302888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724acc0-bd57-4123-a6b0-27063a7fce62 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:48.304384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724acc0-bd57-4123-a6b0-27063a7fce62 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:48.306122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 134/268] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:48 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:48.312100 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:48.428997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe237d46-cd35-437e-8d5c-334e1b28d12e None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.429304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe237d46-cd35-437e-8d5c-334e1b28d12e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.429465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe237d46-cd35-437e-8d5c-334e1b28d12e None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.431119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 135/269] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:48.451832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282acd8-fcd7-4430-82a9-d357cdec4a05 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.453317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282acd8-fcd7-4430-82a9-d357cdec4a05 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.454051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282acd8-fcd7-4430-82a9-d357cdec4a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.469888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f0e5d-4b51-468e-8435-b2c19c9d61a3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:48.475853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:48.490702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c282acd8-fcd7-4430-82a9-d357cdec4a05 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:48.500466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c282acd8-fcd7-4430-82a9-d357cdec4a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:48.551513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f660a59-8040-406f-9825-1f399b78623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:48.552193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660a59-8040-406f-9825-1f399b78623b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.552473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660a59-8040-406f-9825-1f399b78623b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.552743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660a59-8040-406f-9825-1f399b78623b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.553605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f660a59-8040-406f-9825-1f399b78623b None admin] RBAC: Adding query filter params (role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=9bd63e581e0e4653a129d2db83456e19) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:48.557279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f660a59-8040-406f-9825-1f399b78623b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:48.557279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f660a59-8040-406f-9825-1f399b78623b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:48.567880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 135/270] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:48 2022] GET /identity/v3/role_assignments?user.id=9bd63e581e0e4653a129d2db83456e19&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:48.632453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 136/271] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:48.645884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fbcc3-340a-4b07-8389-764be15c625e None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.646216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fbcc3-340a-4b07-8389-764be15c625e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.646465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fbcc3-340a-4b07-8389-764be15c625e None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.647858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 136/272] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:48.653763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a17c77-2427-4130-9b79-9bc6da39f37a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.654044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a17c77-2427-4130-9b79-9bc6da39f37a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.654350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a17c77-2427-4130-9b79-9bc6da39f37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.660111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fbcc3-340a-4b07-8389-764be15c625e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:48.673113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-99a17c77-2427-4130-9b79-9bc6da39f37a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:48.674364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-99a17c77-2427-4130-9b79-9bc6da39f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:48.700752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:48.773052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 137/273] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:48.774573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6dd1-0e43-4901-9399-9e4e5b55f434 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.774865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6dd1-0e43-4901-9399-9e4e5b55f434 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.775144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6dd1-0e43-4901-9399-9e4e5b55f434 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.776751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 138/274] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:48.792389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3850d4-9f86-4af6-bec1-9f6599e8be55 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.792715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3850d4-9f86-4af6-bec1-9f6599e8be55 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.792954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3850d4-9f86-4af6-bec1-9f6599e8be55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.805771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fbcc3-340a-4b07-8389-764be15c625e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:48.821365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-bd3850d4-9f86-4af6-bec1-9f6599e8be55 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:48.823904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bd3850d4-9f86-4af6-bec1-9f6599e8be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:48.850232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:48.851050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:48.851339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:48.851612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:48.852852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:48.854489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:48.856672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 137/275] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:48.857577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:48.901744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:48.916618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 139/276] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:48.917574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3850d4-9f86-4af6-bec1-9f6599e8be55 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:48.975726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96180e4c-c089-4223-9aec-d9cc3212c847 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.012705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.013524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.013808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.014086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.019742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:49.025003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:49.031475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3850d4-9f86-4af6-bec1-9f6599e8be55 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:49.033168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:49.034018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:49.034836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 138/277] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:49.035463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.073478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.074162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.074469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.074761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.075454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.076912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.137372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:49.144062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83d8ae-c3f1-4c01-91d3-4720b532e0d2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.174203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 140/278] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:34:48 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:49.174203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.203629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.205383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.206182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.206470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.225144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:49.236046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:49.236977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:49.237740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:49.238356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 139/279] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:49.244920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.324009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7e3da-26f7-4b68-ad9c-3d5dac3d20aa None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.373815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549d5d0-785b-43c7-8c2b-eb94d02a7394 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.375878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.376659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.377017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.377363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.378221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:49.378596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.380069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.392662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 141/280] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:49.398117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.408689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:49.421343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.427630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.427630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.427630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.427630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:49.427630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.427630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.435227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 140/281] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:49.443196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.465373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:49.543430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42788d04-23b1-40f6-a531-4ea7155f7162 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.580318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2745f-e007-4a5a-adb4-9b94bb8bf2c5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.586157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.587564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.587564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.587564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] PATH_INFO: `/v3/users/e787c2fb2bc0489896fc066fc1a4591d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.588798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] RBAC: Authorizing `identity:get_user(user_id=e787c2fb2bc0489896fc066fc1a4591d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.590311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.592266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 142/282] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:49 2022] GET /identity/v3/users/e787c2fb2bc0489896fc066fc1a4591d => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:49.599685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.631263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.632700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.632986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.633266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] PATH_INFO: `/v3/users/e700d137569f49c08d6d2fe5b86de69a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.651312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] RBAC: Authorizing `identity:get_user(user_id=e700d137569f49c08d6d2fe5b86de69a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.653343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.655978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 141/283] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:49 2022] GET /identity/v3/users/e700d137569f49c08d6d2fe5b86de69a => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:49.662288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.688228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:49.716071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768408e-b87d-461f-8ad9-0c39ccec97f3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.749297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:49.754708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3882-b67d-4098-a207-bfdd7e05462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.755592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3882-b67d-4098-a207-bfdd7e05462b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.755882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3882-b67d-4098-a207-bfdd7e05462b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.756161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3882-b67d-4098-a207-bfdd7e05462b None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.757505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd3882-b67d-4098-a207-bfdd7e05462b None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.759034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd3882-b67d-4098-a207-bfdd7e05462b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.760774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 143/284] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:49 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:49.762292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c512-c3d0-4340-9769-91fac6bd498d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.762591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c512-c3d0-4340-9769-91fac6bd498d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.762872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c512-c3d0-4340-9769-91fac6bd498d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.764577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 144/285] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:49.769036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c512-c3d0-4340-9769-91fac6bd498d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:49.777345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2f371-8c5f-46ab-beec-c1fcfe7c7481 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.819619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178cab1-f3a2-49c7-bbe8-558f53cb1183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.820213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178cab1-f3a2-49c7-bbe8-558f53cb1183 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.820497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178cab1-f3a2-49c7-bbe8-558f53cb1183 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.821025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178cab1-f3a2-49c7-bbe8-558f53cb1183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.822461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178cab1-f3a2-49c7-bbe8-558f53cb1183 None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, user.id=e700d137569f49c08d6d2fe5b86de69a, scope.system=all) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:49.822786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178cab1-f3a2-49c7-bbe8-558f53cb1183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.824850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178cab1-f3a2-49c7-bbe8-558f53cb1183 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.832624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 142/286] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Jun 15 14:34:49 2022] GET /identity/v3/role_assignments?user.id=e700d137569f49c08d6d2fe5b86de69a&scope.system=all&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:49.835153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acee32-d1c3-4d51-b793-53fcb87fb466 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.835448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acee32-d1c3-4d51-b793-53fcb87fb466 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.835732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acee32-d1c3-4d51-b793-53fcb87fb466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.871977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-62acee32-d1c3-4d51-b793-53fcb87fb466 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:49.873389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-62acee32-d1c3-4d51-b793-53fcb87fb466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:49.915599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c512-c3d0-4340-9769-91fac6bd498d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:49.975291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:49.976218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:49.976501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:49.976776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/e787c2fb2bc0489896fc066fc1a4591d/roles/60619ffdb13e417e9263be4069ea17cd` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:49.980100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] RBAC: Authorizing `identity:create_grant(project_id=14d7888970604c63a01c4de4b27aa60c, user_id=e787c2fb2bc0489896fc066fc1a4591d, role_id=60619ffdb13e417e9263be4069ea17cd)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:49.981597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:49.999512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 143/287] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:50.024782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acee32-d1c3-4d51-b793-53fcb87fb466 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.072628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 145/288] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:34:49 2022] PUT /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c/users/e787c2fb2bc0489896fc066fc1a4591d/roles/60619ffdb13e417e9263be4069ea17cd => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:50.111921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.149743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acee32-d1c3-4d51-b793-53fcb87fb466 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.188344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.189169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.189451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.189730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.201608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:50.210391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:50.211709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:50.212461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:50.216939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 144/289] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:50.218473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.221993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.360203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b61fe0-85dd-41f8-a1bd-56d0b0f03d2b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.403933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.404630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.404911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.405188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.405959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] RBAC: Adding query filter params (name=member) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:50.407687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:50.409134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:50.419002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 146/290] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:50.425517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.606340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1ef6c-e9c0-459a-9243-cae2c89ae1b6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.644752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.657281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.658479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.658768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.659045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] PATH_INFO: `/v3/users/f459dd0493ed420090bc0c7698a35da6` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:50.660324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] RBAC: Authorizing `identity:get_user(user_id=f459dd0493ed420090bc0c7698a35da6)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:50.668322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:50.668322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 145/291] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/users/f459dd0493ed420090bc0c7698a35da6 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:50.669810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:50.850538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:50.907895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff95e2-a187-4c63-b0fe-12c736d2a30f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:50.960121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:50.960794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:50.961351 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:50.961635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.042708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.044204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.126819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:51.164859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 147/292] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:51.170567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.300939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4b20b-4af6-454a-af94-ec5c124c02e3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.341708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ba835-57ec-42fb-846d-d423f18af47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.342617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ba835-57ec-42fb-846d-d423f18af47d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.342903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ba835-57ec-42fb-846d-d423f18af47d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.343182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ba835-57ec-42fb-846d-d423f18af47d None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.345946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ba835-57ec-42fb-846d-d423f18af47d None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.347450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ba835-57ec-42fb-846d-d423f18af47d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.349435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 146/293] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:51 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:51.359360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:51.503610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d3ab8-3f4e-49a4-8a3c-158dafc4518a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:51.552140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3445090-1adc-4cf6-87ee-41780c37c107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:51.552851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3445090-1adc-4cf6-87ee-41780c37c107 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.553141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3445090-1adc-4cf6-87ee-41780c37c107 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.553416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3445090-1adc-4cf6-87ee-41780c37c107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.558190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3445090-1adc-4cf6-87ee-41780c37c107 None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=f459dd0493ed420090bc0c7698a35da6) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:51.558190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3445090-1adc-4cf6-87ee-41780c37c107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:51.558190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3445090-1adc-4cf6-87ee-41780c37c107 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:51.567095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 148/294] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:51 2022] GET /identity/v3/role_assignments?user.id=f459dd0493ed420090bc0c7698a35da6&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:51.947383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bce59-7dfa-410f-b84e-a0b6842226ac None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.947750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bce59-7dfa-410f-b84e-a0b6842226ac None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.947949 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bce59-7dfa-410f-b84e-a0b6842226ac None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.949514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 147/295] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:51.955081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dd180-b651-418e-9a04-cf3d0ef21b6a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:51.955377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dd180-b651-418e-9a04-cf3d0ef21b6a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:51.955656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dd180-b651-418e-9a04-cf3d0ef21b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:51.976326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-451dd180-b651-418e-9a04-cf3d0ef21b6a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:51.977530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-451dd180-b651-418e-9a04-cf3d0ef21b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:52.006202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:52.057506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 149/296] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:52.083364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bce59-7dfa-410f-b84e-a0b6842226ac None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.161826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecbc36-292b-4440-bec4-396d0627c307 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.162402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecbc36-292b-4440-bec4-396d0627c307 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.162840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecbc36-292b-4440-bec4-396d0627c307 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.164616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 150/297] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:52.165364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:52.169674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94b63-0a15-413d-8f51-ae31956c9209 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.170267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94b63-0a15-413d-8f51-ae31956c9209 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.170763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94b63-0a15-413d-8f51-ae31956c9209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.188906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f3b94b63-0a15-413d-8f51-ae31956c9209 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:52.190441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f3b94b63-0a15-413d-8f51-ae31956c9209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:52.211004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bce59-7dfa-410f-b84e-a0b6842226ac None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:52.256246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:52.257449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.257932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.258449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.270512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:52.273914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 151/298] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:52.284689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:52.285600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:52.286419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:52.288778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 148/299] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:52.295826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b94b63-0a15-413d-8f51-ae31956c9209 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.297549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.405084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:52.417081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfbb55-73c9-482f-b20f-e571565cdf5d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:52.418119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b94b63-0a15-413d-8f51-ae31956c9209 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:52.442558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:52.454027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81f58f-ae65-4e71-a806-3545b57b1ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:52.454929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81f58f-ae65-4e71-a806-3545b57b1ec2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.455210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81f58f-ae65-4e71-a806-3545b57b1ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.455483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81f58f-ae65-4e71-a806-3545b57b1ec2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.456677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81f58f-ae65-4e71-a806-3545b57b1ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:52.457628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be1a83-45a7-415d-b1d5-82d9c6dc1430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:52.458153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81f58f-ae65-4e71-a806-3545b57b1ec2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:52.458439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be1a83-45a7-415d-b1d5-82d9c6dc1430 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.458722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be1a83-45a7-415d-b1d5-82d9c6dc1430 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.458993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be1a83-45a7-415d-b1d5-82d9c6dc1430 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.460024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be1a83-45a7-415d-b1d5-82d9c6dc1430 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:52.460123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 149/300] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:52.460206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be1a83-45a7-415d-b1d5-82d9c6dc1430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:52.465020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be1a83-45a7-415d-b1d5-82d9c6dc1430 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:52.465184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087bd0a-7ed9-4881-95fd-d9f419e5a2f3 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.465184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087bd0a-7ed9-4881-95fd-d9f419e5a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.465184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087bd0a-7ed9-4881-95fd-d9f419e5a2f3 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.471641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 150/301] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:52.471641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087bd0a-7ed9-4881-95fd-d9f419e5a2f3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.471972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 152/302] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:52.473173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24ff17-5462-4b87-8bb2-a84fa31b7303 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.473461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24ff17-5462-4b87-8bb2-a84fa31b7303 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.473739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24ff17-5462-4b87-8bb2-a84fa31b7303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.494425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1e24ff17-5462-4b87-8bb2-a84fa31b7303 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:52.495678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1e24ff17-5462-4b87-8bb2-a84fa31b7303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:52.581708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 153/303] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:52.583089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24ff17-5462-4b87-8bb2-a84fa31b7303 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.590067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087bd0a-7ed9-4881-95fd-d9f419e5a2f3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:52.629565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:52.630638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.630923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.631326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] PATH_INFO: `/v3/groups` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.631893 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:52.633343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:52.691190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:52.731123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 151/304] 10.222.0.23 () {60 vars in 1134 bytes} [Wed Jun 15 14:34:52 2022] POST /identity/v3/groups => generated 229 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:34:52.731123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.731556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24ff17-5462-4b87-8bb2-a84fa31b7303 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:52.777798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:52.779124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.779416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.779697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] PATH_INFO: `/v3/users/e36c1e09931b4fd6932132cba65d9ecd` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.790302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:52.802464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] RBAC: Authorizing `identity:get_user(user_id=e36c1e09931b4fd6932132cba65d9ecd)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:52.804295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:52.811020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 154/305] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:52 2022] GET /identity/v3/users/e36c1e09931b4fd6932132cba65d9ecd => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:52.812917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.861236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c73238-bdf0-4818-b788-5508baa4187c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:52.907125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:52.907664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.907951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.908227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.921145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:52.929877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:52.931028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:52.932080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:52.934326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 152/306] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:52.942081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:52.942405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9546176-a612-4c89-a701-33fa2ab052f3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:52.995331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4f71d-d3e3-484a-aae6-d7800aeaaf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:52.995916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f71d-d3e3-484a-aae6-d7800aeaaf19 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:52.996195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f71d-d3e3-484a-aae6-d7800aeaaf19 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:52.996467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f71d-d3e3-484a-aae6-d7800aeaaf19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:52.997411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4f71d-d3e3-484a-aae6-d7800aeaaf19 None admin] RBAC: Adding query filter params (role.id=60619ffdb13e417e9263be4069ea17cd, user.id=e36c1e09931b4fd6932132cba65d9ecd, scope.system=all) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:52.997872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4f71d-d3e3-484a-aae6-d7800aeaaf19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:52.999553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4f71d-d3e3-484a-aae6-d7800aeaaf19 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:53.005130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 155/307] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Jun 15 14:34:52 2022] GET /identity/v3/role_assignments?user.id=e36c1e09931b4fd6932132cba65d9ecd&scope.system=all&role.id=60619ffdb13e417e9263be4069ea17cd => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:53.019208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973304-ea91-47e2-bfb4-c3a1b0820a97 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.019681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973304-ea91-47e2-bfb4-c3a1b0820a97 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.020057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973304-ea91-47e2-bfb4-c3a1b0820a97 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.022046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 156/308] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:53.027516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011c48d-a65f-42ee-acb7-853aee23bddb None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.027942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011c48d-a65f-42ee-acb7-853aee23bddb None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.028315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011c48d-a65f-42ee-acb7-853aee23bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.048243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9011c48d-a65f-42ee-acb7-853aee23bddb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:53.049835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9011c48d-a65f-42ee-acb7-853aee23bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:53.099305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c4b36-885a-4891-a326-125e1faf3d0c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.128680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.133409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 157/309] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:53.134728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.151821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.153017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.153516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.154069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.155208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:53.155730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:53.158609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011c48d-a65f-42ee-acb7-853aee23bddb None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:53.159334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:53.170943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 153/310] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:53.176954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:53.295833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011c48d-a65f-42ee-acb7-853aee23bddb None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.316842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209d615-d818-4f9a-aa5f-94b630939f12 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.342344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.348099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.348900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.349183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.349454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.361531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.362481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.362724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:53.362834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.363077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] PATH_INFO: `/v3/users/e700d137569f49c08d6d2fe5b86de69a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.364434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] RBAC: Authorizing `identity:get_user(user_id=e700d137569f49c08d6d2fe5b86de69a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:53.365964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:53.370711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:53.371570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:53.372339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:53.372923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 154/311] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:53 2022] GET /identity/v3/users/e700d137569f49c08d6d2fe5b86de69a => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:53.378197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:53.378348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 158/312] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:53.379972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:53.424675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.493932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210101-9618-4cde-b69d-f83a8a9700e1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.522260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f67fc0-de37-4e5b-9a0e-03fd2c84a02b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.535798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.536921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.537404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.537938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] PATH_INFO: `/v3/system/users/e700d137569f49c08d6d2fe5b86de69a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.539771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e700d137569f49c08d6d2fe5b86de69a, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:53.541433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:53.566050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.566752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.567032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.567317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.568062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:53.568370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:53.569786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:53.580159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 159/313] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:53.586623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:53.650505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 155/314] 10.222.0.23 () {58 vars in 1334 bytes} [Wed Jun 15 14:34:53 2022] PUT /identity/v3/system/users/e700d137569f49c08d6d2fe5b86de69a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:34:53.668633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.726136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ce01d-f274-4c43-b080-5cf7f1a75db1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.773506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.774840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.775354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.775921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] PATH_INFO: `/v3/users/e787c2fb2bc0489896fc066fc1a4591d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:53.777481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] RBAC: Authorizing `identity:get_user(user_id=e787c2fb2bc0489896fc066fc1a4591d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:53.779238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:53.782025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 160/315] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:53 2022] GET /identity/v3/users/e787c2fb2bc0489896fc066fc1a4591d => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:53.790296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:53.910357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e114156-87ad-405c-a711-7a6308f4b28b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:53.919125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:53.952067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:53.953680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:53.954531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:53.955302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.043053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:54.044746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:54.160647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 156/316] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:54.166420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:54.288016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829736e-4db2-462f-a8b3-9c76c55f1435 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:54.357767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:54.370341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd206689-c1ae-46ee-87a0-1eb6cdfd2be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:54.371213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206689-c1ae-46ee-87a0-1eb6cdfd2be6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.371501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206689-c1ae-46ee-87a0-1eb6cdfd2be6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.371780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd206689-c1ae-46ee-87a0-1eb6cdfd2be6 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.373050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd206689-c1ae-46ee-87a0-1eb6cdfd2be6 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:54.374590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd206689-c1ae-46ee-87a0-1eb6cdfd2be6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:54.377335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 161/317] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:54 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:54.390350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:54.393710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:54.496200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150afe2-1a94-4dea-af8d-06fec9488185 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:54.535606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262490e4-b7e6-434d-81b6-701c82e9df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:54.536563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262490e4-b7e6-434d-81b6-701c82e9df0a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:54.537087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262490e4-b7e6-434d-81b6-701c82e9df0a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:54.537665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262490e4-b7e6-434d-81b6-701c82e9df0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:54.538763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262490e4-b7e6-434d-81b6-701c82e9df0a None admin] RBAC: Adding query filter params (role.id=60619ffdb13e417e9263be4069ea17cd, scope.project.id=14d7888970604c63a01c4de4b27aa60c, user.id=e787c2fb2bc0489896fc066fc1a4591d) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:54.539302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262490e4-b7e6-434d-81b6-701c82e9df0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:54.540971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262490e4-b7e6-434d-81b6-701c82e9df0a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:54.548556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 157/318] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:34:54 2022] GET /identity/v3/role_assignments?user.id=e787c2fb2bc0489896fc066fc1a4591d&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=60619ffdb13e417e9263be4069ea17cd => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:55.113928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62294929-d31a-4e6b-a386-852be3f9afd0 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.114480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62294929-d31a-4e6b-a386-852be3f9afd0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.114480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62294929-d31a-4e6b-a386-852be3f9afd0 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.116056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 162/319] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:55.121100 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52391-5f0e-4ec1-9f39-878061c2f817 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.121396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52391-5f0e-4ec1-9f39-878061c2f817 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.121663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52391-5f0e-4ec1-9f39-878061c2f817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.139473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-62e52391-5f0e-4ec1-9f39-878061c2f817 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:55.140753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-62e52391-5f0e-4ec1-9f39-878061c2f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:55.216065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.222846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 158/320] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:55.228368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6240f09-338b-469b-b82c-561c82b54e1a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.228721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6240f09-338b-469b-b82c-561c82b54e1a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.229072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6240f09-338b-469b-b82c-561c82b54e1a None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.230775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 163/321] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:55.236324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcec54-bc0a-4595-acb2-3b499104b40b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.242119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcec54-bc0a-4595-acb2-3b499104b40b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.242119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcec54-bc0a-4595-acb2-3b499104b40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.246488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6240f09-338b-469b-b82c-561c82b54e1a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.255029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e6bcec54-bc0a-4595-acb2-3b499104b40b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:55.256231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e6bcec54-bc0a-4595-acb2-3b499104b40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:55.328799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 159/322] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:55.351324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcec54-bc0a-4595-acb2-3b499104b40b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.371646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6240f09-338b-469b-b82c-561c82b54e1a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.408915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.409718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.410011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.410320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.411633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.413224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.415202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 164/323] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:34:55.420643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.460329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcec54-bc0a-4595-acb2-3b499104b40b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.494455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.499343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.502630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.503602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.504992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.504992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.518355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:55.528725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:55.529737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:34:55.531166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 160/324] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:34:55.536756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.553111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2be904-bf63-4def-9622-8554a4d59ee2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.592204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.592873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.593157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.593432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] PATH_INFO: `/v3/groups` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.594216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.595758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.647195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dd07e-350d-435c-9abf-3c8d9709b05e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.681940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.682638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.682922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.683201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.683951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:55.684261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.685662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.700153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 161/325] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:55.700354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 165/326] 10.222.0.23 () {60 vars in 1134 bytes} [Wed Jun 15 14:34:55 2022] POST /identity/v3/groups => generated 226 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:34:55.705759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:55.769834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:55.825311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7def8ab-4ef7-4ea2-81fc-cad899e90c33 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:55.864548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd623a6-9957-4edd-a4da-c498ee401d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:55.865441 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd623a6-9957-4edd-a4da-c498ee401d1a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:55.865724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd623a6-9957-4edd-a4da-c498ee401d1a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:55.866000 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd623a6-9957-4edd-a4da-c498ee401d1a None admin] PATH_INFO: `/v3/users/e36c1e09931b4fd6932132cba65d9ecd` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:55.868084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd623a6-9957-4edd-a4da-c498ee401d1a None admin] RBAC: Authorizing `identity:get_user(user_id=e36c1e09931b4fd6932132cba65d9ecd)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:55.869572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd623a6-9957-4edd-a4da-c498ee401d1a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:55.871424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 162/327] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:55 2022] GET /identity/v3/users/e36c1e09931b4fd6932132cba65d9ecd => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:55.877620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.004239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dcdfd-1f17-4952-87b2-cad955190582 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.014736 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e33f7d-1471-44f5-a685-a4e32333e6f3 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.015051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e33f7d-1471-44f5-a685-a4e32333e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.015312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e33f7d-1471-44f5-a685-a4e32333e6f3 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.017908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 163/328] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:34:56.023842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424af79-ed36-4c0e-8b71-a85f213581c3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.024128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424af79-ed36-4c0e-8b71-a85f213581c3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.024400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424af79-ed36-4c0e-8b71-a85f213581c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.027340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:56.043268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-d424af79-ed36-4c0e-8b71-a85f213581c3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:56.047944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d424af79-ed36-4c0e-8b71-a85f213581c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:56.052300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:56.053162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.053441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.053713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] PATH_INFO: `/v3/system/users/e36c1e09931b4fd6932132cba65d9ecd/roles/60619ffdb13e417e9263be4069ea17cd` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.055280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e36c1e09931b4fd6932132cba65d9ecd, role_id=60619ffdb13e417e9263be4069ea17cd)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:56.057762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:56.115230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 166/329] 10.222.0.23 () {58 vars in 1334 bytes} [Wed Jun 15 14:34:55 2022] PUT /identity/v3/system/users/e36c1e09931b4fd6932132cba65d9ecd/roles/60619ffdb13e417e9263be4069ea17cd => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:34:56.155060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 164/330] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:34:56.179581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.321254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1bad-6d34-4e91-b68c-3d6c1b790cd0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.360684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:56.361508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.361791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.362068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.374515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:56.384355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:56.385277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:56.386300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 167/331] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:56.391594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424af79-ed36-4c0e-8b71-a85f213581c3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.450401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:56.512506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424af79-ed36-4c0e-8b71-a85f213581c3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.555907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:56.555907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.555907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.555907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.556303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:56.556595 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:56.557998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:56.573178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 165/332] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:56.574517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.658228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:56.743763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718109-d897-4d5d-9079-ff81f0fb3331 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.790880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c48128-c037-4ad4-8b6b-e60189eef196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:56.792151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c48128-c037-4ad4-8b6b-e60189eef196 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:56.792649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c48128-c037-4ad4-8b6b-e60189eef196 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:56.793113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c48128-c037-4ad4-8b6b-e60189eef196 None admin] PATH_INFO: `/v3/users/e700d137569f49c08d6d2fe5b86de69a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:56.799164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c48128-c037-4ad4-8b6b-e60189eef196 None admin] RBAC: Authorizing `identity:get_user(user_id=e700d137569f49c08d6d2fe5b86de69a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:56.800950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c48128-c037-4ad4-8b6b-e60189eef196 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:56.803265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 168/333] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:34:56 2022] GET /identity/v3/users/e700d137569f49c08d6d2fe5b86de69a => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:56.808804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:56.946136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a551b-8239-47b5-9348-0e266da8923d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:56.968153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:57.002726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce5032-e9c3-4f07-b3ae-6050b927bedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:57.003410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce5032-e9c3-4f07-b3ae-6050b927bedb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:57.003697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce5032-e9c3-4f07-b3ae-6050b927bedb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:57.003976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce5032-e9c3-4f07-b3ae-6050b927bedb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:57.005221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce5032-e9c3-4f07-b3ae-6050b927bedb None admin] RBAC: Adding query filter params (role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, user.id=e700d137569f49c08d6d2fe5b86de69a, scope.system=all) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:57.005597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce5032-e9c3-4f07-b3ae-6050b927bedb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:57.007196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce5032-e9c3-4f07-b3ae-6050b927bedb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:57.026380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 166/334] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Jun 15 14:34:56 2022] GET /identity/v3/role_assignments?user.id=e700d137569f49c08d6d2fe5b86de69a&scope.system=all&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:58.610970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20849a-76a6-441c-b4c6-70d068a6f96b None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.611930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20849a-76a6-441c-b4c6-70d068a6f96b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.612398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20849a-76a6-441c-b4c6-70d068a6f96b None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.614005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 169/335] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:58.627948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99d77d-9445-4a71-9a35-ebb8fbe25bad None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.628451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99d77d-9445-4a71-9a35-ebb8fbe25bad None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.628522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99d77d-9445-4a71-9a35-ebb8fbe25bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.657296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6f99d77d-9445-4a71-9a35-ebb8fbe25bad None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:58.658540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6f99d77d-9445-4a71-9a35-ebb8fbe25bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:58.746992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 167/336] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:58.791928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20849a-76a6-441c-b4c6-70d068a6f96b None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:58.929447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20849a-76a6-441c-b4c6-70d068a6f96b None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:58.941870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:58.972368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00cf7-9230-4a5c-966f-021335f79490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:58.973553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00cf7-9230-4a5c-966f-021335f79490 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:58.974094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00cf7-9230-4a5c-966f-021335f79490 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:58.974674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00cf7-9230-4a5c-966f-021335f79490 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:58.991933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a00cf7-9230-4a5c-966f-021335f79490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-c0a00cf7-9230-4a5c-966f-021335f79490 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:59.000743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:59.001756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:34:59.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:59.004979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 170/337] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:59.010898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99d77d-9445-4a71-9a35-ebb8fbe25bad None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.175204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99d77d-9445-4a71-9a35-ebb8fbe25bad None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.178530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.183080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a10ee-3094-422c-b8a5-e72ea86e76b0 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.184409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a10ee-3094-422c-b8a5-e72ea86e76b0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.184905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a10ee-3094-422c-b8a5-e72ea86e76b0 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.188959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 171/338] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:59.193932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1adf4-033a-4b33-8989-ea7da9019e87 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.214320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1adf4-033a-4b33-8989-ea7da9019e87 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.214320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1adf4-033a-4b33-8989-ea7da9019e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.220431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.221123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.221412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.221691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.228254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:34:59.228254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.228254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.233498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 168/339] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:59.235871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-15d1adf4-033a-4b33-8989-ea7da9019e87 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:59.238201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-15d1adf4-033a-4b33-8989-ea7da9019e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:59.243493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.365016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 172/340] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:59.388254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba644e-e798-4cdf-8723-ffb48c1d52c3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.407536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1adf4-033a-4b33-8989-ea7da9019e87 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.454185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003076f7-dd76-474e-86e4-2498d14a9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.459139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003076f7-dd76-474e-86e4-2498d14a9a77 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.459457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003076f7-dd76-474e-86e4-2498d14a9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.459785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003076f7-dd76-474e-86e4-2498d14a9a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.511329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.553007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003076f7-dd76-474e-86e4-2498d14a9a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.554566 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003076f7-dd76-474e-86e4-2498d14a9a77 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.616222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1adf4-033a-4b33-8989-ea7da9019e87 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.623989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.667779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 169/341] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:34:59.669678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38387-a2c7-4cf8-93cb-d9800e24abf0 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.670117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38387-a2c7-4cf8-93cb-d9800e24abf0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.670561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f38387-a2c7-4cf8-93cb-d9800e24abf0 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.672731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 170/342] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:34:59.679829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e130ac9-04ac-4e83-91d6-48e433b27819 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.680401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e130ac9-04ac-4e83-91d6-48e433b27819 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.681046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e130ac9-04ac-4e83-91d6-48e433b27819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.699882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8e130ac9-04ac-4e83-91d6-48e433b27819 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:34:59.701541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8e130ac9-04ac-4e83-91d6-48e433b27819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:34:59.727019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.728101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.728567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.729033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.767270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:34:59.770556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:34:59.782323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 171/343] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:34:59.783592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e130ac9-04ac-4e83-91d6-48e433b27819 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:34:59.784685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:34:59.785750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 173/344] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:34:59.809230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.903050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e130ac9-04ac-4e83-91d6-48e433b27819 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:34:59.943274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:34:59.944391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:34:59.944841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:34:59.945275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:34:59.946663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:34:59.948396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:34:59.950576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 172/345] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:34:59.951661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:34:59.980162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d653583-8d20-4f81-94af-7eeeed4724aa None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.019752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.022976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31125d6f-5bae-466b-a783-53f5dcaebbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.023898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125d6f-5bae-466b-a783-53f5dcaebbca None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.024334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125d6f-5bae-466b-a783-53f5dcaebbca None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.024765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125d6f-5bae-466b-a783-53f5dcaebbca None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.025675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31125d6f-5bae-466b-a783-53f5dcaebbca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:00.026126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31125d6f-5bae-466b-a783-53f5dcaebbca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.027709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31125d6f-5bae-466b-a783-53f5dcaebbca None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.038448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 174/346] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:00.040045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074945-4306-4417-8928-8a354b4eb000 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.040501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074945-4306-4417-8928-8a354b4eb000 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.040936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074945-4306-4417-8928-8a354b4eb000 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.042393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 175/347] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:00.043218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074945-4306-4417-8928-8a354b4eb000 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.064303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa989186-eb67-42a1-9eda-b2ba0c075eb2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.076054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.104557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.105408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.106024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.106488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.119710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:00.128178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:00.129302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 14:35:00.130323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:00.132444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 173/348] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:34:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:00.133384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.171059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074945-4306-4417-8928-8a354b4eb000 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.207993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb24007-93a1-4e19-8138-00696307881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.209158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24007-93a1-4e19-8138-00696307881f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.209644 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24007-93a1-4e19-8138-00696307881f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.210105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24007-93a1-4e19-8138-00696307881f None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.218012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb24007-93a1-4e19-8138-00696307881f None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.220566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb24007-93a1-4e19-8138-00696307881f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.222692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 176/349] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:00.224093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa8230-e0f7-4baf-9201-152e9f1a5a42 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.224575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa8230-e0f7-4baf-9201-152e9f1a5a42 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.229983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa8230-e0f7-4baf-9201-152e9f1a5a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.247890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c01c6-f83c-4de8-baa5-9cfb6fdd8b02 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.248755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-68aa8230-e0f7-4baf-9201-152e9f1a5a42 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:00.250216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-68aa8230-e0f7-4baf-9201-152e9f1a5a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:00.269950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.286571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.287273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.287801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.288237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.327124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 177/350] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:00.328895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa8230-e0f7-4baf-9201-152e9f1a5a42 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.357471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.367460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.370153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.466778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa8230-e0f7-4baf-9201-152e9f1a5a42 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.487799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 174/351] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:00.488872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.505853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.506961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.506961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.507192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.508014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:00.508607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.510352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.513796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.519770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 178/352] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:00.520672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.618289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd4557-9346-4017-ac98-255e7efb7c25 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.625987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.657357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fc27f-2f96-4273-992d-469ff30c6ad9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.673317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.673996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.674336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.674613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.675464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:00.675775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.677259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.687962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 175/353] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:00.688486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.713567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72b984-6262-4ffa-affe-b832c2a89039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.714840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b984-6262-4ffa-affe-b832c2a89039 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.715363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b984-6262-4ffa-affe-b832c2a89039 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.717870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b984-6262-4ffa-affe-b832c2a89039 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.733118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72b984-6262-4ffa-affe-b832c2a89039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-5b72b984-6262-4ffa-affe-b832c2a89039 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:00.743935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:00.744876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:00.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:00.750082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 179/354] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:00.754132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a3c8f-5d61-40b6-b693-21d1b891119a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.754132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a3c8f-5d61-40b6-b693-21d1b891119a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.754132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a3c8f-5d61-40b6-b693-21d1b891119a None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.756331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 180/355] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:00.757054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a3c8f-5d61-40b6-b693-21d1b891119a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.815439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:00.835381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de8042-c88d-4dc0-92c0-17e6b87cbc5e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.883841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a30661-6de3-4f19-a907-99647048862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.884480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a30661-6de3-4f19-a907-99647048862c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.884763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a30661-6de3-4f19-a907-99647048862c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.885036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a30661-6de3-4f19-a907-99647048862c None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.886331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a30661-6de3-4f19-a907-99647048862c None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.889274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a30661-6de3-4f19-a907-99647048862c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.894655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 176/356] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:00.895786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a30661-6de3-4f19-a907-99647048862c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:00.927779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a3c8f-5d61-40b6-b693-21d1b891119a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:00.985299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9bfc4-cecb-4029-8b7f-129d40e38152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:00.986362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9bfc4-cecb-4029-8b7f-129d40e38152 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.986605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9bfc4-cecb-4029-8b7f-129d40e38152 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.986884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9bfc4-cecb-4029-8b7f-129d40e38152 None admin] PATH_INFO: `/v3/users/e36c1e09931b4fd6932132cba65d9ecd` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:00.988422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9bfc4-cecb-4029-8b7f-129d40e38152 None admin] RBAC: Authorizing `identity:get_user(user_id=e36c1e09931b4fd6932132cba65d9ecd)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:00.990098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9bfc4-cecb-4029-8b7f-129d40e38152 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:00.993060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 181/357] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/users/e36c1e09931b4fd6932132cba65d9ecd => generated 293 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:00.994858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6982c-ee3c-4aeb-aa37-4cf06a48ea53 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:00.995152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6982c-ee3c-4aeb-aa37-4cf06a48ea53 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:00.995432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6982c-ee3c-4aeb-aa37-4cf06a48ea53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.011964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:01.018730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1ab6982c-ee3c-4aeb-aa37-4cf06a48ea53 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:01.020012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1ab6982c-ee3c-4aeb-aa37-4cf06a48ea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:01.053371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a30661-6de3-4f19-a907-99647048862c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:01.113513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 182/358] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:01.115173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6982c-ee3c-4aeb-aa37-4cf06a48ea53 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:01.116252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:01.116944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.117230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.117509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.118387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:01.118703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:01.120959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:01.130313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 177/359] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:35:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:01.131174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:01.151944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:01.260977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c49cd-1e21-4903-b2da-dae061623af6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:01.261211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6982c-ee3c-4aeb-aa37-4cf06a48ea53 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:01.306806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:01.307612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.307898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.308177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.309417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:01.310980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:01.311727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:01.312747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.317820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 183/360] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:01.317820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:01.318014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.318014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.318014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] RBAC: Adding query filter params (role.id=60619ffdb13e417e9263be4069ea17cd, user.id=e36c1e09931b4fd6932132cba65d9ecd, scope.system=all) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:01.318014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:01.318456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:01.321427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:01.336681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 178/361] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/role_assignments?user.id=e36c1e09931b4fd6932132cba65d9ecd&scope.system=all&role.id=60619ffdb13e417e9263be4069ea17cd => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:01.337758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:01.434191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:01.450161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd87135-72d6-4fc5-9da6-0ac6bd61f85c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:01.472389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701534c1-3d88-4cd4-ace4-ba77671deb03 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:01.490058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084696a-77f0-46d1-87d7-9c03cc3e66e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:01.490969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084696a-77f0-46d1-87d7-9c03cc3e66e4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.491356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084696a-77f0-46d1-87d7-9c03cc3e66e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.491847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084696a-77f0-46d1-87d7-9c03cc3e66e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.514674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5b75e-0295-46e2-be31-71d79f3edde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:01.516381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5b75e-0295-46e2-be31-71d79f3edde3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.516815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5b75e-0295-46e2-be31-71d79f3edde3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.517239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5b75e-0295-46e2-be31-71d79f3edde3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.529793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5b75e-0295-46e2-be31-71d79f3edde3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-38c5b75e-0295-46e2-be31-71d79f3edde3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:01.539437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:01.540363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:01.541144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:01.543666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 179/362] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:01.545206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e630-a972-41cd-a02c-b5f60789b5c4 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.545708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e630-a972-41cd-a02c-b5f60789b5c4 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.546194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e630-a972-41cd-a02c-b5f60789b5c4 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.548417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 180/363] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:01.549393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e630-a972-41cd-a02c-b5f60789b5c4 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:01.577209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084696a-77f0-46d1-87d7-9c03cc3e66e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:01.579474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084696a-77f0-46d1-87d7-9c03cc3e66e4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:01.583303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:01.687878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e630-a972-41cd-a02c-b5f60789b5c4 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:01.711767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 184/364] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:01.713099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe52034-c12e-4c96-a899-d02a7061c500 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.713425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe52034-c12e-4c96-a899-d02a7061c500 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.713659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe52034-c12e-4c96-a899-d02a7061c500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.731164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3fe52034-c12e-4c96-a899-d02a7061c500 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:01.732744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3fe52034-c12e-4c96-a899-d02a7061c500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:01.737137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:01.743742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:01.744595 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.744916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.745197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.746547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:01.746920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:01.748686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:01.756119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 181/365] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:01.756847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:01.837741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 185/366] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:01.839133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe52034-c12e-4c96-a899-d02a7061c500 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:01.924771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc683c-e059-49ff-9381-3ea18178e3d6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:01.938944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:01.978375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:01.979070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:01.979357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:01.979635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:01.980402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:01.980714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:01.983394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:01.994790 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 182/367] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:01.995704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.026367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe52034-c12e-4c96-a899-d02a7061c500 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.072300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.073179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.073523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.073967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.075364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:02.076855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:02.079458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 186/368] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:02.080349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.085054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.141937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1f74f-9549-40b5-927f-a4a077481835 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.181880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.183053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.183554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.184020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.200103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:02.201233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd076cc-58c5-414a-bbf5-e4219e9943ad None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:02.210934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:02.212105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:02.212900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:02.214497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.221639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 183/369] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:02.221639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.268676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.269762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.271469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.271469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.360765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923464d6-3c79-4756-9be2-2e3ebee7f352 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.365404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:02.367789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:02.388399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.410502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.411295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.411578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.411850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.413033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:02.416288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:02.418257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 184/370] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:02.419033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.504779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 187/371] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:02.505699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.508357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.556741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f0447-c3ec-4ad9-8f93-ccae5ccb86b6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.618958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.619327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.619642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.619918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.620677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:02.620987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:02.622573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:02.635283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 185/372] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:02.635283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.650436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28822189-32b4-4c4a-9742-ced496adc70c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.690418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.691386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.691875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.693254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.693715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.694760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=14d7888970604c63a01c4de4b27aa60c) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:02.695278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:02.697015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:02.704217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 188/373] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:02.705340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.787377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec5bf5-78bc-4142-bc25-ef43b51b38eb None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.823673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:02.828051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.829825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.830108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.831466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.832823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:02.834346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:02.836127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 186/374] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:02.842045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:02.892857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81efaa3-11e8-44a5-853e-8f0d7f6c0ab0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:02.963987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:02.964582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:02.964864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:02.965138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:02.986632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2f134-6ea3-42d4-af7b-5628b91f493b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.022715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.054331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.055265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.055552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.055828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.058086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.059852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.061643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 187/375] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:03.069092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.104143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.105641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.194709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.276855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 189/376] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:03.282403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.336035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d419-ee76-4fd5-924e-aeb72016954e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.419777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d02bf-5a6b-4900-b27d-a2cd0754d2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.420937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d02bf-5a6b-4900-b27d-a2cd0754d2b9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.421884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d02bf-5a6b-4900-b27d-a2cd0754d2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.422430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d02bf-5a6b-4900-b27d-a2cd0754d2b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.424319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d02bf-5a6b-4900-b27d-a2cd0754d2b9 None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=14d7888970604c63a01c4de4b27aa60c) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:03.424909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d02bf-5a6b-4900-b27d-a2cd0754d2b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.427340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d02bf-5a6b-4900-b27d-a2cd0754d2b9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.439372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 188/377] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 260 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:03.440766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137af82-bfa3-4e91-8bbd-8e23e0502163 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.441052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137af82-bfa3-4e91-8bbd-8e23e0502163 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.441332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137af82-bfa3-4e91-8bbd-8e23e0502163 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.448575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75184650-d8f9-4077-ad25-1e1cf88fe0e2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.450297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 189/378] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:03.467873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd435-5f1e-421b-bc61-7b4f2483c914 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.468306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd435-5f1e-421b-bc61-7b4f2483c914 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.468518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd435-5f1e-421b-bc61-7b4f2483c914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.497244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.498127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.498431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.498704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] PATH_INFO: `/v3/projects/108eb4dc74a441578e74c932122b6873` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.501667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] RBAC: Authorizing `identity:get_project(project_id=108eb4dc74a441578e74c932122b6873)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.503174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.508509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 190/379] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/projects/108eb4dc74a441578e74c932122b6873 => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:03.509710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a2fbd435-5f1e-421b-bc61-7b4f2483c914 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:03.515069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a2fbd435-5f1e-421b-bc61-7b4f2483c914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:03.518950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.585921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.622528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.669170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 190/380] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:03.669961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8c03-c88e-4901-82b8-0c6e12e307a3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.702972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbd435-5f1e-421b-bc61-7b4f2483c914 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.716019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.716902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.717181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.717458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] PATH_INFO: `/v3/groups/881292e9bd8c493daae66bfe44b43b22` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.723437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] RBAC: Authorizing `identity:get_group(group_id=881292e9bd8c493daae66bfe44b43b22)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.724922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.726995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 191/381] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/groups/881292e9bd8c493daae66bfe44b43b22 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:03.732908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:03.849836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:03.854235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457f7d2-0ef4-4de9-85da-812ff329d660 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.860840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbd435-5f1e-421b-bc61-7b4f2483c914 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:03.915669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.916882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.917313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.917915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.919560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] RBAC: Adding query filter params (group.id=881292e9bd8c493daae66bfe44b43b22, role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=108eb4dc74a441578e74c932122b6873) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:03.920046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:03.922522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:03.923864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:03.924738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:03.925017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:03.925290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:03.930273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 192/382] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/role_assignments?group.id=881292e9bd8c493daae66bfe44b43b22&scope.project.id=108eb4dc74a441578e74c932122b6873&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:03.956497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:03.964531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:03.965623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:03.966599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:03.970677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 191/383] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:03.978573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.158195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bbe4b-14cb-4593-9d88-a58f731de572 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.201601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.202498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.202753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.202989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.203736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:04.204041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.205476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.215864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 193/384] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:04.222112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.274111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:04.354657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a5296-9624-46ec-8ebd-87e339801581 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.403223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.404090 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.404410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.404689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.406578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.408319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.410206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 192/385] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:04 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:04.415748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.449863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:04.548348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777ca4a-d41a-4648-b97b-b97924b3aac6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.613027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3b3dc-6766-49a5-a600-fa35b880f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.614299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b3dc-6766-49a5-a600-fa35b880f714 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.614725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b3dc-6766-49a5-a600-fa35b880f714 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.615141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b3dc-6766-49a5-a600-fa35b880f714 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.618093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3b3dc-6766-49a5-a600-fa35b880f714 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.620537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3b3dc-6766-49a5-a600-fa35b880f714 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.623813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 194/386] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:04 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:04.633362 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:04.785936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac4656-36b8-40dd-931c-e26ffa6da903 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:04.795662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:04.812545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aafb2-629d-47cd-bdc6-ccc054059102 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.812545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aafb2-629d-47cd-bdc6-ccc054059102 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.819980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aafb2-629d-47cd-bdc6-ccc054059102 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.819980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 195/387] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:04.833106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:04.833292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f87ff-113c-47a0-9d6d-4a20804b08be None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.833535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f87ff-113c-47a0-9d6d-4a20804b08be None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.833822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f87ff-113c-47a0-9d6d-4a20804b08be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.833994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:04.834311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:04.834592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/groups/843c668da1b642eca61b24a3451306af/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:04.839034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74553fbb1344338e5312c0d2bd5d4e, group_id=843c668da1b642eca61b24a3451306af, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:04.840697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:04.856760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b47f87ff-113c-47a0-9d6d-4a20804b08be None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:04.857973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b47f87ff-113c-47a0-9d6d-4a20804b08be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:04.928275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 193/388] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Jun 15 14:35:04 2022] PUT /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/groups/843c668da1b642eca61b24a3451306af/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:04.960650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 196/389] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:05.003377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:05.150064 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea4d9e-6c23-4f01-a031-6d49b09a71f7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:05.201334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:05.203491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.204057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.204591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.216876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:05.226069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:05.231701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:05.232472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 194/390] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:35:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:05.235380 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f87ff-113c-47a0-9d6d-4a20804b08be None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:05.277734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:05.377318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f87ff-113c-47a0-9d6d-4a20804b08be None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:05.424734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:05.426052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.426844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.427477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.428746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:05.429255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:05.430948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:05.449534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 197/391] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:05.457020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:05.503965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:05.611347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94985ff-e73a-43cf-92ad-c28978e6bea3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:05.668917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58dc70-8b2e-4e4e-ab6d-bfa564ce53b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:05.669803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc70-8b2e-4e4e-ab6d-bfa564ce53b1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.670089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc70-8b2e-4e4e-ab6d-bfa564ce53b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.670397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc70-8b2e-4e4e-ab6d-bfa564ce53b1 None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.675254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58dc70-8b2e-4e4e-ab6d-bfa564ce53b1 None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:05.676659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58dc70-8b2e-4e4e-ab6d-bfa564ce53b1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:05.678853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 195/392] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:05 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:05.684369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:05.821607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:05.841724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca9e9-36ca-4280-b028-59efc6390023 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:05.855091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e7fb-d06e-4502-a5e3-5c8f72613bc0 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.855576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e7fb-d06e-4502-a5e3-5c8f72613bc0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.855676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e7fb-d06e-4502-a5e3-5c8f72613bc0 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.857171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 196/393] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:05.866054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5e7ba-383e-4da9-b75b-a745743ed302 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.866685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5e7ba-383e-4da9-b75b-a745743ed302 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.867202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5e7ba-383e-4da9-b75b-a745743ed302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.895027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:05.899464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:05.899968 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:05.900440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:05.902186 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-97a5e7ba-383e-4da9-b75b-a745743ed302 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:05.903071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:05.904468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-97a5e7ba-383e-4da9-b75b-a745743ed302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:05.908358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:05.911433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 198/394] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:05 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:05.920683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:06.042000 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 197/395] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:06.071718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5e7ba-383e-4da9-b75b-a745743ed302 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:06.120232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a344e19-844c-4878-b496-f28a69e914f6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:06.120871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:06.199698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:06.200611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.200898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.201181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/groups/843c668da1b642eca61b24a3451306af/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.208530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74553fbb1344338e5312c0d2bd5d4e, group_id=843c668da1b642eca61b24a3451306af, role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:06.210024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:06.261503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5e7ba-383e-4da9-b75b-a745743ed302 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:06.309174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167b874-3123-49c0-89c5-b00186310b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:06.309947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167b874-3123-49c0-89c5-b00186310b23 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.310269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167b874-3123-49c0-89c5-b00186310b23 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.310550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167b874-3123-49c0-89c5-b00186310b23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.324593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 199/396] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Jun 15 14:35:05 2022] PUT /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e/groups/843c668da1b642eca61b24a3451306af/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:06.332046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167b874-3123-49c0-89c5-b00186310b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-c167b874-3123-49c0-89c5-b00186310b23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:06.340681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:06.341642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:06.342471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 198/397] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:06.349752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:06.354021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690853d9-caee-4acb-8d5a-ee0270c96c30 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.354388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690853d9-caee-4acb-8d5a-ee0270c96c30 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.354623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690853d9-caee-4acb-8d5a-ee0270c96c30 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.356271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 199/398] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:06.362351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529ee0-5489-48a4-8768-873e0f0ed2e6 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.362686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529ee0-5489-48a4-8768-873e0f0ed2e6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.362919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529ee0-5489-48a4-8768-873e0f0ed2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.375002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:06.398922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-cb529ee0-5489-48a4-8768-873e0f0ed2e6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:06.400146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cb529ee0-5489-48a4-8768-873e0f0ed2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:06.537967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 200/399] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:06.562642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e63a-235c-423d-b401-940cfe9d8c06 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:06.578764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb529ee0-5489-48a4-8768-873e0f0ed2e6 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:06.600218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:06.600969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.601248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.601520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.602313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:06.602621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:06.604155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:06.609114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:06.615272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 200/400] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:06.624696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:06.757300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb529ee0-5489-48a4-8768-873e0f0ed2e6 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:06.757923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:06.810788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4182fb3-cbd9-482a-8bcb-878ebdd6c7d5 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:06.823158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:06.823905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.824186 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.824461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.851219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:06.871642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:06.873359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:06.874221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 201/401] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:06.882799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:06.883628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:06.884650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:06.885523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:06.885822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:06.887664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:06.889176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:06.899702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 201/402] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:06.900895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.014273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.041861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efb176-3c2f-4586-8bf7-b852bc5640a1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.066397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85f9d1-a67f-4fd6-aa92-484751a2cbf0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.081732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd4bd6-8810-4ae0-83e2-fe66882df83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.082464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd4bd6-8810-4ae0-83e2-fe66882df83c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.082821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd4bd6-8810-4ae0-83e2-fe66882df83c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.083172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd4bd6-8810-4ae0-83e2-fe66882df83c None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.084244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd4bd6-8810-4ae0-83e2-fe66882df83c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:07.084358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd4bd6-8810-4ae0-83e2-fe66882df83c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.085807 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd4bd6-8810-4ae0-83e2-fe66882df83c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.100154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 202/403] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:07.101817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b58f1-3744-4169-8810-e4e728924488 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.102122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b58f1-3744-4169-8810-e4e728924488 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.102577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b58f1-3744-4169-8810-e4e728924488 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.108843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 203/404] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:07.108986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0f066-73f6-495f-8bb2-40aabed8adc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.109720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f066-73f6-495f-8bb2-40aabed8adc4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.110004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f066-73f6-495f-8bb2-40aabed8adc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.110321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f066-73f6-495f-8bb2-40aabed8adc4 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.110559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b58f1-3744-4169-8810-e4e728924488 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.111541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0f066-73f6-495f-8bb2-40aabed8adc4 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.113008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0f066-73f6-495f-8bb2-40aabed8adc4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.116656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 202/405] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:06 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:07.118003 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e83d2-93bb-4266-ba0b-3cfe31315c6e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.118338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e83d2-93bb-4266-ba0b-3cfe31315c6e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.118619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e83d2-93bb-4266-ba0b-3cfe31315c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.140881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-0e5e83d2-93bb-4266-ba0b-3cfe31315c6e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:07.142378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0e5e83d2-93bb-4266-ba0b-3cfe31315c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:07.147162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.242674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 203/406] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:07.244017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e83d2-93bb-4266-ba0b-3cfe31315c6e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.248456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b58f1-3744-4169-8810-e4e728924488 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.286497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.287358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.287697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.287977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.290887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.292049 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.299943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 204/407] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:07.301613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.305165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.392052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e83d2-93bb-4266-ba0b-3cfe31315c6e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.409544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67389-6b8b-48dc-ac85-89c342e053d8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.433402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.434698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.435153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.435653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c/groups/843c668da1b642eca61b24a3451306af/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.438752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] RBAC: Authorizing `identity:create_grant(project_id=14d7888970604c63a01c4de4b27aa60c, group_id=843c668da1b642eca61b24a3451306af, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.440726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.444957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.445901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.446211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.446500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.447764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.449159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.451074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 205/408] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:07.454286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.537588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.557037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.563989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 204/409] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Jun 15 14:35:07 2022] PUT /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c/groups/843c668da1b642eca61b24a3451306af/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:07.565436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.597990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd65f-8b00-47d2-85cd-cd8671cbb0b0 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.674490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.675596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.675596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.675860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.703888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:07.712581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea0a6-66a7-4fca-9693-8a7efec24a8b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:07.720295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:07.721352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:07.722133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 206/410] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:07.728462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:07.765815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8db724-530c-4856-add6-1bf6df7b4f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.767699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db724-530c-4856-add6-1bf6df7b4f1b None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.768218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db724-530c-4856-add6-1bf6df7b4f1b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.769048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db724-530c-4856-add6-1bf6df7b4f1b None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c/groups/843c668da1b642eca61b24a3451306af/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.771702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8db724-530c-4856-add6-1bf6df7b4f1b None admin] RBAC: Authorizing `identity:create_grant(project_id=14d7888970604c63a01c4de4b27aa60c, group_id=843c668da1b642eca61b24a3451306af, role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.773804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8db724-530c-4856-add6-1bf6df7b4f1b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.852189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 205/411] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Jun 15 14:35:07 2022] PUT /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c/groups/843c668da1b642eca61b24a3451306af/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:07.854484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdf01a-8ad4-4eee-8863-52ef410439ac None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.854773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdf01a-8ad4-4eee-8863-52ef410439ac None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.855051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdf01a-8ad4-4eee-8863-52ef410439ac None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.860906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 206/412] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:07.870450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301420a-847a-461b-af5b-33bcd674e3d7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.870815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301420a-847a-461b-af5b-33bcd674e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.871176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301420a-847a-461b-af5b-33bcd674e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.879960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fb148-11fc-4fec-b7b0-8665479207fb None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:07.912695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b301420a-847a-461b-af5b-33bcd674e3d7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:07.914028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b301420a-847a-461b-af5b-33bcd674e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:07.920035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:07.934576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:07.935753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:07.936308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:07.937007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:07.938704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:07.939872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:07.946653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:07.968180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 207/413] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:07.970765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.041490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.058514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 207/414] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:08.084643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301420a-847a-461b-af5b-33bcd674e3d7 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.100438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc31c3d-8b47-46ca-b8ef-cb6984fe725b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.135548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.136437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.136689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.136962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] PATH_INFO: `/v3/groups/881292e9bd8c493daae66bfe44b43b22` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.138156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] RBAC: Authorizing `identity:get_group(group_id=881292e9bd8c493daae66bfe44b43b22)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.140258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.142677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 208/415] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:07 2022] GET /identity/v3/groups/881292e9bd8c493daae66bfe44b43b22 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:08.148205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.219033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301420a-847a-461b-af5b-33bcd674e3d7 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.265877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.266894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.267270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.267551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.272228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.291680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:08.300314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:08.301491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:08.302504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:08.310890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 208/416] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:08.312692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.327768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78838ba-e4fb-4818-ae00-34c5a0e292cc None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.369436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.370624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.371103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.371596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] PATH_INFO: `/v3/projects/108eb4dc74a441578e74c932122b6873` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.373292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] RBAC: Authorizing `identity:get_project(project_id=108eb4dc74a441578e74c932122b6873)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.377518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.380415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 209/417] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/projects/108eb4dc74a441578e74c932122b6873 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:08.395232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.403444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.480144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c2e62-9a8e-4742-9422-4727d8c7de6a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.521824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.523712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.524393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.524843 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.525765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] RBAC: Adding query filter params (name=member) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:08.526257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.528589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.551103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 209/418] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:08.557249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:08.629849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea31b23-b54a-4c29-bbe8-b47e5f5f086d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.666817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb6859-9054-4def-aecb-4dea1884057b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:08.680769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.695285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:08.706119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedd3fa-1765-4b22-a0b6-f8842556622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.706804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedd3fa-1765-4b22-a0b6-f8842556622a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.707092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedd3fa-1765-4b22-a0b6-f8842556622a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.707369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedd3fa-1765-4b22-a0b6-f8842556622a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.709213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:08.710147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:08.710480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:08.711009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] PATH_INFO: `/v3/projects/108eb4dc74a441578e74c932122b6873/groups/881292e9bd8c493daae66bfe44b43b22/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:08.714950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=108eb4dc74a441578e74c932122b6873, group_id=881292e9bd8c493daae66bfe44b43b22, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.716403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.794149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedd3fa-1765-4b22-a0b6-f8842556622a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:08.796188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedd3fa-1765-4b22-a0b6-f8842556622a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:08.891202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 210/419] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Jun 15 14:35:08 2022] PUT /identity/v3/projects/108eb4dc74a441578e74c932122b6873/groups/881292e9bd8c493daae66bfe44b43b22/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:08.916709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 210/420] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:08.923974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.049105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d396-d4f0-4ba0-95da-257e17b30a25 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.049612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d396-d4f0-4ba0-95da-257e17b30a25 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.050029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d396-d4f0-4ba0-95da-257e17b30a25 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.051899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 211/421] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:09.059144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a66f5-c67c-4424-9e31-106fb2cec546 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.059569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a66f5-c67c-4424-9e31-106fb2cec546 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.060010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a66f5-c67c-4424-9e31-106fb2cec546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.081798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43984549-5581-4107-bcee-73fb743307f8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:09.103232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-5e0a66f5-c67c-4424-9e31-106fb2cec546 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:09.104455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5e0a66f5-c67c-4424-9e31-106fb2cec546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:09.118829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:09.122245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:09.123122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.123398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.123693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.125741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:09.127322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:09.129479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 211/422] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:08 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:09.137205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.202807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 212/423] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:09.228177 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:09.246033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a66f5-c67c-4424-9e31-106fb2cec546 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.287296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9467d-3ff0-42bd-947b-0087d153c4ce None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:09.361339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:09.363609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.363932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.364164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.365637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:09.367175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:09.368909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 212/424] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:09 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:09.379820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.457998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a66f5-c67c-4424-9e31-106fb2cec546 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:09.504321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:09.505954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.506284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.506756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.523981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:09.534737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:09.535623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:09.536384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 213/425] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:35:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:09.542496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.546334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:09.638600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e03208-86ef-4cfc-8d79-1f95ae00760b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:09.690451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:09.703613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:09.704308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.704593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.704867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.705716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:09.706039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:09.707532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:09.721672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 213/426] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:09 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 611 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:09.733243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950a1d-1e8a-4d88-8eaf-e0119e3a4e52 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:09.788785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:09.789869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:09.790403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:09.790881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:09.791858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:09.792396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:09.794077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:09.809221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 214/427] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:09.816292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:09.988354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc3a04-d2ef-4cd7-bf16-b92ee3c867b1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:10.026046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f94505-9bbe-4007-bf2d-98c2f1a7e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:10.027113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f94505-9bbe-4007-bf2d-98c2f1a7e803 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.027596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f94505-9bbe-4007-bf2d-98c2f1a7e803 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.028070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f94505-9bbe-4007-bf2d-98c2f1a7e803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.094946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:10.106428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f94505-9bbe-4007-bf2d-98c2f1a7e803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:10.109010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f94505-9bbe-4007-bf2d-98c2f1a7e803 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:10.219756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 214/428] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:10.224911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:10.298710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:10.335937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd711a-3bd3-4d0e-b813-51224992f5d8 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.336504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd711a-3bd3-4d0e-b813-51224992f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.337162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd711a-3bd3-4d0e-b813-51224992f5d8 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.339677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 215/429] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:10.345968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a602-723b-4e84-962a-707adc25bb6e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.346336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a602-723b-4e84-962a-707adc25bb6e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.346569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a602-723b-4e84-962a-707adc25bb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.346663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c08e88-bfd3-481a-8908-a79955d5b2b4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:10.381034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7b36a602-723b-4e84-962a-707adc25bb6e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:10.382618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7b36a602-723b-4e84-962a-707adc25bb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:10.393042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:10.394374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.394887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.395391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] PATH_INFO: `/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.397693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74553fbb1344338e5312c0d2bd5d4e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:10.399715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:10.402954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 215/430] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/projects/cc74553fbb1344338e5312c0d2bd5d4e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:10.408896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:10.482528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 216/431] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:10.503275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a602-723b-4e84-962a-707adc25bb6e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:10.532983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:10.538997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cd50f-8d0c-4e2d-9cd1-db821c08dff6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:10.581589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:10.591389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:10.592789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.593475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.593967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.595681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:10.597434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:10.600047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 216/432] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:10.606340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:10.649045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a602-723b-4e84-962a-707adc25bb6e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:10.700906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8f2bc-85c8-4488-b4d3-ec77d2675921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:10.702320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f2bc-85c8-4488-b4d3-ec77d2675921 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.702741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f2bc-85c8-4488-b4d3-ec77d2675921 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.703021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8f2bc-85c8-4488-b4d3-ec77d2675921 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.716717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8f2bc-85c8-4488-b4d3-ec77d2675921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-0df8f2bc-85c8-4488-b4d3-ec77d2675921 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:10.726296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:10.727164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 14:35:10.727930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:10.728509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 217/433] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:10.729510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf10cb-17f5-4736-9467-69c1e4a957b0 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.729800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf10cb-17f5-4736-9467-69c1e4a957b0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.730134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf10cb-17f5-4736-9467-69c1e4a957b0 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.731970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 218/434] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:10.733371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf10cb-17f5-4736-9467-69c1e4a957b0 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:10.757749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bffadb-187e-4f00-913a-974a76e2262f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:10.815877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e173-fc64-4cd1-bdc0-394c792d9323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:10.817484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e173-fc64-4cd1-bdc0-394c792d9323 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.817971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e173-fc64-4cd1-bdc0-394c792d9323 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.818699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e173-fc64-4cd1-bdc0-394c792d9323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.819955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e173-fc64-4cd1-bdc0-394c792d9323 None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:10.821285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e173-fc64-4cd1-bdc0-394c792d9323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:10.822966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e173-fc64-4cd1-bdc0-394c792d9323 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:10.831970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 217/435] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=cc74553fbb1344338e5312c0d2bd5d4e&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:10.832653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:10.835159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04b70b-7259-4655-813d-8c90e30e8182 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.835903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04b70b-7259-4655-813d-8c90e30e8182 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.836368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04b70b-7259-4655-813d-8c90e30e8182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.855242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf10cb-17f5-4736-9467-69c1e4a957b0 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:10.862505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ae04b70b-7259-4655-813d-8c90e30e8182 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:10.864787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ae04b70b-7259-4655-813d-8c90e30e8182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:10.899038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:10.900160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:10.900623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:10.901077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:10.902051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] RBAC: Adding query filter params (name=member) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:10.902569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:10.904285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:10.914334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 219/436] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:10.921359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:10.949394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:10.966484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 218/437] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:10.985317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04b70b-7259-4655-813d-8c90e30e8182 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.056019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f23e86-76b8-440e-9d55-ec60e74c2127 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.104741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.104741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.104741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.104741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.111465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04b70b-7259-4655-813d-8c90e30e8182 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.118750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.155491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.157064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.157064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.157064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.170891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:11.179166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:11.180077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:11.180850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 219/438] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:11.186935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.202924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.205908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.269858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.328464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac1b06-3b49-47fb-ae93-6cab88584cd9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.334553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 220/439] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:11.340819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.367672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.368275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.368565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.368848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.369605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:11.369918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.371353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.380559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 220/440] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:11.385627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.392702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.455909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89f1cf-0316-407c-8c69-e525a529b241 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.490868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.491714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.491993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.492264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.493435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.494909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.496569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 221/441] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:11.502525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.504316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230af439-9e1a-4357-bda5-add5f22f8283 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.522918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.550848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfdc1e-6f29-46e1-aaaa-b992d84ecb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.552352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfdc1e-6f29-46e1-aaaa-b992d84ecb16 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.552661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfdc1e-6f29-46e1-aaaa-b992d84ecb16 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.552948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfdc1e-6f29-46e1-aaaa-b992d84ecb16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.622826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfdc1e-6f29-46e1-aaaa-b992d84ecb16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.624855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfdc1e-6f29-46e1-aaaa-b992d84ecb16 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.625448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1427f-23c1-496b-bc3f-2d9bb33f03e4 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.636314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.660250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff73a4-160c-43c0-b774-1900bc569602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.661097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff73a4-160c-43c0-b774-1900bc569602 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.661376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff73a4-160c-43c0-b774-1900bc569602 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.661647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff73a4-160c-43c0-b774-1900bc569602 None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.662810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff73a4-160c-43c0-b774-1900bc569602 None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.664262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff73a4-160c-43c0-b774-1900bc569602 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.666188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 222/442] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:11.667479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9523-9d40-4cde-9d9c-34e1128739be None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.668094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9523-9d40-4cde-9d9c-34e1128739be None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.668370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9523-9d40-4cde-9d9c-34e1128739be None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.669576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 223/443] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:11.671600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9523-9d40-4cde-9d9c-34e1128739be None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.728223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 221/444] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:35:11.729600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd353a-671f-474c-821f-b22ef030fe2a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.729890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd353a-671f-474c-821f-b22ef030fe2a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.730192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd353a-671f-474c-821f-b22ef030fe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.747115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-56cd353a-671f-474c-821f-b22ef030fe2a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:11.748322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-56cd353a-671f-474c-821f-b22ef030fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:11.759604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.781006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9523-9d40-4cde-9d9c-34e1128739be None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.818551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 222/445] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:11.825998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd353a-671f-474c-821f-b22ef030fe2a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.828285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.829304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.829598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.829879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.831086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=926cbd7dfa704847bbfbf9d0c2aab5fc, scope.project.id=14d7888970604c63a01c4de4b27aa60c) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:11.831603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.833066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.841957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 224/446] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=926cbd7dfa704847bbfbf9d0c2aab5fc => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:11.843110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:11.897445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:11.936697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd353a-671f-474c-821f-b22ef030fe2a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.967985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430faee7-2ef6-44e5-899c-2d97214e8ada None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:11.974915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:11.975803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:11.976080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:11.976345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] PATH_INFO: `/v3/projects/14d7888970604c63a01c4de4b27aa60c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:11.977665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] RBAC: Authorizing `identity:get_project(project_id=14d7888970604c63a01c4de4b27aa60c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:11.979182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:11.986528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 223/447] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/projects/14d7888970604c63a01c4de4b27aa60c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:11.988033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.005990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:12.010502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.011296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.011577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.011905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.022948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:12.030709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:12.031615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:12.032793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 225/448] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:12.038194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.112695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc66bc1-713f-422b-9350-16cf9a48cbf5 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.154037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.155573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.155869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.156142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] PATH_INFO: `/v3/groups/843c668da1b642eca61b24a3451306af` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.157374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] RBAC: Authorizing `identity:get_group(group_id=843c668da1b642eca61b24a3451306af)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.158876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:12.160668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 224/449] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:11 2022] GET /identity/v3/groups/843c668da1b642eca61b24a3451306af => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:12.165977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f5d32-a0ee-49c6-9071-2f520b9df3db None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.167711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:12.167839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.205959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.207095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.207095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.207297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.207950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:12.208257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.209649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:12.221726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 226/450] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:12.227552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.261635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:12.280790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c40f00-91f0-4543-aeae-e096223f28e0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.318523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.319182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.320438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.320718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.321560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] RBAC: Adding query filter params (group.id=843c668da1b642eca61b24a3451306af, role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=14d7888970604c63a01c4de4b27aa60c) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:12.321870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.323362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:12.329831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 225/451] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:12 2022] GET /identity/v3/role_assignments?group.id=843c668da1b642eca61b24a3451306af&scope.project.id=14d7888970604c63a01c4de4b27aa60c&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:12.340160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc996-4bc9-445b-8f44-91bb2d9d8771 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.377018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.377693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.377978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.378284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.412653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:12.453524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.455278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:12.559882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 227/452] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:35:12.566414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.668478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1adaedb-daf6-47ab-8c61-b07da75012c8 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.703641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.704504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.704785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.705059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] PATH_INFO: `/v3/projects/108eb4dc74a441578e74c932122b6873` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.706256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] RBAC: Authorizing `identity:get_project(project_id=108eb4dc74a441578e74c932122b6873)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.707708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:12.709382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 226/453] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:35:12 2022] GET /identity/v3/projects/108eb4dc74a441578e74c932122b6873 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:12.715207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.774727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:12.810508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d4c92-10ac-427b-b7ac-255302dcffcd None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.844548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581f4c3-5180-474e-8b50-8bcfc320095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.845392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f4c3-5180-474e-8b50-8bcfc320095e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.845670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f4c3-5180-474e-8b50-8bcfc320095e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.845939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f4c3-5180-474e-8b50-8bcfc320095e None admin] PATH_INFO: `/v3/groups/881292e9bd8c493daae66bfe44b43b22` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.847118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581f4c3-5180-474e-8b50-8bcfc320095e None admin] RBAC: Authorizing `identity:get_group(group_id=881292e9bd8c493daae66bfe44b43b22)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.848550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581f4c3-5180-474e-8b50-8bcfc320095e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:12.850147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 228/454] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Jun 15 14:35:12 2022] GET /identity/v3/groups/881292e9bd8c493daae66bfe44b43b22 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:12.856090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:12.872339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:12.959705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8c80b-29f1-4448-8511-3dc12638ef9d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:12.993396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14ba67-6dcd-4707-95be-dc5b8a720adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:12.994036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14ba67-6dcd-4707-95be-dc5b8a720adc None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:12.994345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14ba67-6dcd-4707-95be-dc5b8a720adc None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:12.994617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14ba67-6dcd-4707-95be-dc5b8a720adc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:12.995439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14ba67-6dcd-4707-95be-dc5b8a720adc None admin] RBAC: Adding query filter params (group.id=881292e9bd8c493daae66bfe44b43b22, role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=108eb4dc74a441578e74c932122b6873) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:12.995747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14ba67-6dcd-4707-95be-dc5b8a720adc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:12.997194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14ba67-6dcd-4707-95be-dc5b8a720adc None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:13.003389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 227/455] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Jun 15 14:35:12 2022] GET /identity/v3/role_assignments?group.id=881292e9bd8c493daae66bfe44b43b22&scope.project.id=108eb4dc74a441578e74c932122b6873&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:15.673424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a42003-a49d-4f18-8dd9-00abd94c3e0b None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:15.674306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a42003-a49d-4f18-8dd9-00abd94c3e0b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:15.674724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a42003-a49d-4f18-8dd9-00abd94c3e0b None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:15.676466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 229/456] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:15.689062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333449d3-b1b5-4261-96c5-d48b6afe9023 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:15.689892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333449d3-b1b5-4261-96c5-d48b6afe9023 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:15.690356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333449d3-b1b5-4261-96c5-d48b6afe9023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:15.712427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-333449d3-b1b5-4261-96c5-d48b6afe9023 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:15.713830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-333449d3-b1b5-4261-96c5-d48b6afe9023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:15.796969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 228/457] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:15.834494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a42003-a49d-4f18-8dd9-00abd94c3e0b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:15.906382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084bfd4-4724-4152-a0c5-1a4c50c1c7df None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:15.906894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084bfd4-4724-4152-a0c5-1a4c50c1c7df None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:15.907309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084bfd4-4724-4152-a0c5-1a4c50c1c7df None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:15.909240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 229/458] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:15.913861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d7263-7bf3-4d2e-9ea4-90266cc13b4f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:15.914387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d7263-7bf3-4d2e-9ea4-90266cc13b4f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:15.914864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d7263-7bf3-4d2e-9ea4-90266cc13b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:15.944053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b57d7263-7bf3-4d2e-9ea4-90266cc13b4f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:15.945261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b57d7263-7bf3-4d2e-9ea4-90266cc13b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:15.947534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.019403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a42003-a49d-4f18-8dd9-00abd94c3e0b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.067334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 230/459] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:16.081533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.092815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.098798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.098798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.098798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.099025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a532b-e17e-48e3-949b-ec1c3877fee4 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.099551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a532b-e17e-48e3-949b-ec1c3877fee4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.100030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a532b-e17e-48e3-949b-ec1c3877fee4 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.103009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 231/460] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:16.107363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9b7ff-d090-4a08-8843-83a93051b734 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.108544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9b7ff-d090-4a08-8843-83a93051b734 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.109030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9b7ff-d090-4a08-8843-83a93051b734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.110615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.112099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.126787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-4ad9b7ff-d090-4a08-8843-83a93051b734 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:16.128480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4ad9b7ff-d090-4a08-8843-83a93051b734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:16.137763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:16.138690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:16.140091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:16.140091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:16.140091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:16.140091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:16.140091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 230/461] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:16.140579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.214260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 232/462] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:16.215694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9b7ff-d090-4a08-8843-83a93051b734 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.262333 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51e1a7-78d9-4241-932a-e7a46041b797 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.309285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.315111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.315923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.316202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.316473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.332038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.334165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.348489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9b7ff-d090-4a08-8843-83a93051b734 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:16.361529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:16.362428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:16.363260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:16.363260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:16.363260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:16.363260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:16.369936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 231/463] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:16.369936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.398369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.399251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.399540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.400155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.401183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:16.401494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.403714 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.407921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.431537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 233/464] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:16.431537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.474933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b8462-2860-434e-9183-8db604d0d8de None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.510473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a822224-2c05-42b4-aab3-b1efd04e295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.511263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a822224-2c05-42b4-aab3-b1efd04e295d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.511548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a822224-2c05-42b4-aab3-b1efd04e295d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.511826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a822224-2c05-42b4-aab3-b1efd04e295d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.522768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a822224-2c05-42b4-aab3-b1efd04e295d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.524213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a822224-2c05-42b4-aab3-b1efd04e295d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-1a822224-2c05-42b4-aab3-b1efd04e295d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:16.542212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:16.543026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:16.543842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:16.543842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:16.543842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:16.543842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:16.544069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184997f5-29cf-4d37-85b6-744677488b70 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.544140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 232/465] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:16.545135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc368be4-7f13-444b-b284-4abf8d157520 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.545465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc368be4-7f13-444b-b284-4abf8d157520 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.545742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc368be4-7f13-444b-b284-4abf8d157520 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.547156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 233/466] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:16.551599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc368be4-7f13-444b-b284-4abf8d157520 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.579159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.587296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793355a-799c-41da-a303-9190e1297833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.587906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793355a-799c-41da-a303-9190e1297833 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.588186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793355a-799c-41da-a303-9190e1297833 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.588466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793355a-799c-41da-a303-9190e1297833 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.589191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793355a-799c-41da-a303-9190e1297833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:16.589501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793355a-799c-41da-a303-9190e1297833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.590933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793355a-799c-41da-a303-9190e1297833 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.602092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 234/467] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:16.602784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793355a-799c-41da-a303-9190e1297833 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.653973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.668344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc368be4-7f13-444b-b284-4abf8d157520 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.705613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.706444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.706596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.706684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793355a-799c-41da-a303-9190e1297833 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:16.707022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.707563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.709033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.740061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0f31b-eb0e-452d-9275-b2092ebbf552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:16.740726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0f31b-eb0e-452d-9275-b2092ebbf552 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.741010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0f31b-eb0e-452d-9275-b2092ebbf552 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.741294 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0f31b-eb0e-452d-9275-b2092ebbf552 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.742010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0f31b-eb0e-452d-9275-b2092ebbf552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:16.742358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0f31b-eb0e-452d-9275-b2092ebbf552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:16.743753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0f31b-eb0e-452d-9275-b2092ebbf552 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:16.756132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 235/468] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:16.757433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b21a4-25ff-4b1f-9a3e-466d7639ed3a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:16.757726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b21a4-25ff-4b1f-9a3e-466d7639ed3a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:16.758004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b21a4-25ff-4b1f-9a3e-466d7639ed3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:16.775726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d14b21a4-25ff-4b1f-9a3e-466d7639ed3a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:16.776908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d14b21a4-25ff-4b1f-9a3e-466d7639ed3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:16.820029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:16.854278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 236/469] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:16.854278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b21a4-25ff-4b1f-9a3e-466d7639ed3a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.863953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 234/470] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:35:16 2022] POST /identity/v3/users => generated 246 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:16.865531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:16.973215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b21a4-25ff-4b1f-9a3e-466d7639ed3a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.018252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baeee1-b20f-4a19-9aba-c3ba393be544 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.028526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.029782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.030071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.030377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.031583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.033657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.059095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cad094-bba9-4cfa-abfb-8c30a021cbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.059721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cad094-bba9-4cfa-abfb-8c30a021cbdb None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.063832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cad094-bba9-4cfa-abfb-8c30a021cbdb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.063832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cad094-bba9-4cfa-abfb-8c30a021cbdb None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.063832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cad094-bba9-4cfa-abfb-8c30a021cbdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.063832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cad094-bba9-4cfa-abfb-8c30a021cbdb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.129851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 237/471] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:35:16 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:17.130864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:17.152423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:17.164382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 235/472] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:35:16 2022] POST /identity/v3/users => generated 248 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:17.177172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:17.274244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7109c0f-397c-4613-8489-3d5de73d43db None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.307373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b957aa-ed43-45bc-85a6-0814937b0c2d None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.307651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b957aa-ed43-45bc-85a6-0814937b0c2d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.307950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b957aa-ed43-45bc-85a6-0814937b0c2d None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.309651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 236/473] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:17.316852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852394-cd05-4a1e-9d1e-f167062c29b9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.317243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852394-cd05-4a1e-9d1e-f167062c29b9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.317452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852394-cd05-4a1e-9d1e-f167062c29b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.327073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.327917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.328272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.328556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.343477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.344231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1b852394-cd05-4a1e-9d1e-f167062c29b9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:17.345118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.346443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1b852394-cd05-4a1e-9d1e-f167062c29b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:17.368283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:17.369194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:17.370038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:17.370038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:17.370038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:17.370038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:17.370038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 238/474] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:17.381801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:17.490851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 237/475] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:17.509986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:17.530821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7d62c-17bc-4b6f-8b43-a366240eba35 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.539694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b852394-cd05-4a1e-9d1e-f167062c29b9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:17.571835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.572510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.573065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.573350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.574085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:17.574441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.576943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.596855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 239/476] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:17.599608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:17.686226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b852394-cd05-4a1e-9d1e-f167062c29b9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.719684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:17.723712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.724672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.725156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.725631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.726612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.728212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.734153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fbd27-2851-433f-b234-cda3f73c73b9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.762381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.sql.core [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:35:17.762381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:35:17.762381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [parameters: {'id': 'affb239d6faf44f188017e8e364391eb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:35:17.762381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87001) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application result.read() Jun 15 14:35:17.784200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:35:17.785144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application yield resource Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise self.value Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application session.commit() Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:35:17.786075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:35:17.786940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:35:17.787779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application result.read() Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:35:17.788585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [parameters: {'id': 'affb239d6faf44f188017e8e364391eb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:17.789423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 14:35:17.791679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:17.804065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 238/477] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Jun 15 14:35:17 2022] POST /identity/v3/roles => generated 144 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:17.811323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:17.814758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.820252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.820252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.820252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.820252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:17.824797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:17.934804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 240/478] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:35:17 2022] POST /identity/v3/users => generated 249 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:17.940466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc602f-2556-4620-b30f-5f1ceeba4708 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:17.979688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c491-6662-42bf-8048-54a4c05389cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:17.980516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c491-6662-42bf-8048-54a4c05389cb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:17.980884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c491-6662-42bf-8048-54a4c05389cb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:17.981161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c491-6662-42bf-8048-54a4c05389cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:17.993829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416c491-6662-42bf-8048-54a4c05389cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-c416c491-6662-42bf-8048-54a4c05389cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:18.002322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:18.003265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:18.004266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 239/479] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:35:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:18.011168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:18.020395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:18.188606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20745ba-bf93-4871-b8cf-795325883871 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:18.238000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc8a8d-f98e-43a4-a444-11140c1ff371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:18.238649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8a8d-f98e-43a4-a444-11140c1ff371 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:18.238931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8a8d-f98e-43a4-a444-11140c1ff371 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:18.239205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8a8d-f98e-43a4-a444-11140c1ff371 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:18.239951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8a8d-f98e-43a4-a444-11140c1ff371 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:18.240256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8a8d-f98e-43a4-a444-11140c1ff371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:18.241650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8a8d-f98e-43a4-a444-11140c1ff371 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:18.251845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 241/480] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:19.769216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b539a-7a97-4fbf-9d62-99ccaeb6147e None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.770275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b539a-7a97-4fbf-9d62-99ccaeb6147e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.770876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b539a-7a97-4fbf-9d62-99ccaeb6147e None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.773047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 240/481] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:19.778269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd52bd1-10b9-458e-8f6d-4b420289d1ee None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.779694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd52bd1-10b9-458e-8f6d-4b420289d1ee None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.780171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd52bd1-10b9-458e-8f6d-4b420289d1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.805066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ccd52bd1-10b9-458e-8f6d-4b420289d1ee None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:19.807045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ccd52bd1-10b9-458e-8f6d-4b420289d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:19.893812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee9037-854e-4f87-a772-2f1804e330ae None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.893812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee9037-854e-4f87-a772-2f1804e330ae None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.893812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee9037-854e-4f87-a772-2f1804e330ae None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.893812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 241/482] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:19.898118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:19.899123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a71248-023a-48d2-b962-b6eeaa5808b3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:19.899403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a71248-023a-48d2-b962-b6eeaa5808b3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:19.899697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a71248-023a-48d2-b962-b6eeaa5808b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:19.912634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 242/483] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:19.936608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd52bd1-10b9-458e-8f6d-4b420289d1ee None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:19.939264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b7a71248-023a-48d2-b962-b6eeaa5808b3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:19.941191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b7a71248-023a-48d2-b962-b6eeaa5808b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:20.050379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd52bd1-10b9-458e-8f6d-4b420289d1ee None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.070624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:20.095791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.096610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.096893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.097309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.115332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:20.116136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 242/484] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:20.129902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:20.130871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:20.131652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:20.135071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 243/485] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:20.141227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a71248-023a-48d2-b962-b6eeaa5808b3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.154707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.219622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:20.271875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc9eb8-765b-4a81-ae2a-991c2c050559 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.330955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35867bc-add8-4e86-a33e-996773302a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.332655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35867bc-add8-4e86-a33e-996773302a69 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.333397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35867bc-add8-4e86-a33e-996773302a69 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.334119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35867bc-add8-4e86-a33e-996773302a69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.338229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a71248-023a-48d2-b962-b6eeaa5808b3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.354462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35867bc-add8-4e86-a33e-996773302a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-d35867bc-add8-4e86-a33e-996773302a69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:20.371800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:20.373197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:20.374070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:20.376947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 244/486] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:20.385092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d227-11f6-4a72-83a0-2b818d273146 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.385301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b629d-b7ef-4e62-b0a2-a05fdb338138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.386013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b629d-b7ef-4e62-b0a2-a05fdb338138 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.386379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b629d-b7ef-4e62-b0a2-a05fdb338138 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.386662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b629d-b7ef-4e62-b0a2-a05fdb338138 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.387121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d227-11f6-4a72-83a0-2b818d273146 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.387432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b629d-b7ef-4e62-b0a2-a05fdb338138 None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:20.387745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b629d-b7ef-4e62-b0a2-a05fdb338138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:20.388916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38d227-11f6-4a72-83a0-2b818d273146 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.390086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b629d-b7ef-4e62-b0a2-a05fdb338138 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:20.390881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 245/487] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:20 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:20.391836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38d227-11f6-4a72-83a0-2b818d273146 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.407314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 243/488] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:20.409500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d86f5-12b6-4e6c-bf08-49d78484bdb6 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.410003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d86f5-12b6-4e6c-bf08-49d78484bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.410890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d86f5-12b6-4e6c-bf08-49d78484bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.432090 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-735d86f5-12b6-4e6c-bf08-49d78484bdb6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:20.433984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-735d86f5-12b6-4e6c-bf08-49d78484bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:20.493680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:20.530835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 244/489] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:20.538644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d86f5-12b6-4e6c-bf08-49d78484bdb6 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.540663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38d227-11f6-4a72-83a0-2b818d273146 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.568118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:20.597227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.598930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.599590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.600067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.601086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:20.601611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:20.604347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:20.623430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 246/490] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:20.624748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.697458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d86f5-12b6-4e6c-bf08-49d78484bdb6 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.734582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.735776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.736298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.736898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.747007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce7f0b-6af3-4959-817d-08413a1c1ae1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.749349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:20.751074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:20.770331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:20.771640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:20.772448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:20.772448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:20.772448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:20.772448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:20.774796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 245/491] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:20.776018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.786552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.787818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.787818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.788098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.800981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:20.809309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:20.810524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:20.811815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:20.816218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 247/492] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:20.817150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.822537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:20.860299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:20.893841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faad9c-5c8d-4b67-8a54-411c3a7876a8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.929713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.929953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7b25-99be-4882-b124-90cd8e5cc023 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:20.930823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.931209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.931588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.943569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:20.945344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:20.965339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:20.966488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:20.967491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:20.967491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:20.967491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:20.967491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:20.968660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 246/493] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:20.969670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:20.976363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:20.977215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:20.977620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:20.978002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:20.978908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:20.979323 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:20.980895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:20.996092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 248/494] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:20.997122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.081019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.104016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa6015-3bd3-4898-b0ff-7cd10ca8ccf9 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.110816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.116733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7fd5c-988f-407d-83b1-f2a9da3b42ff None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.140345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.140345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.140345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.140730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.141241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:21.141537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.142985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.151292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 247/495] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:21.152079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.153108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6319e91-e912-4ab2-997c-cfa594b9f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.154532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6319e91-e912-4ab2-997c-cfa594b9f241 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.155040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6319e91-e912-4ab2-997c-cfa594b9f241 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.155375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6319e91-e912-4ab2-997c-cfa594b9f241 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.156433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6319e91-e912-4ab2-997c-cfa594b9f241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:21.156539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6319e91-e912-4ab2-997c-cfa594b9f241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.157926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6319e91-e912-4ab2-997c-cfa594b9f241 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.169142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 249/496] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:21.170846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83f54e-24fb-4e4e-9415-0acc3c549d1e None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.171133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83f54e-24fb-4e4e-9415-0acc3c549d1e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.171435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83f54e-24fb-4e4e-9415-0acc3c549d1e None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.172665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 250/497] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:21.174460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ced317-fab9-4d91-b969-1f554697fc5e None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.174460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ced317-fab9-4d91-b969-1f554697fc5e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.174460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ced317-fab9-4d91-b969-1f554697fc5e None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.175583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 251/498] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:21.176288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ced317-fab9-4d91-b969-1f554697fc5e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.266493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96d212-dc31-42fa-b203-1128b1544e6f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.284897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ced317-fab9-4d91-b969-1f554697fc5e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.302286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.303083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.303361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.303635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.312579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.314041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.320346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01519947-a9b6-43b2-a0b4-999f56ce322a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.321154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01519947-a9b6-43b2-a0b4-999f56ce322a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.321440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01519947-a9b6-43b2-a0b4-999f56ce322a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.321715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01519947-a9b6-43b2-a0b4-999f56ce322a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:21.330145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:21.331081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:21.331938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 248/499] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:21.332503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.333753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01519947-a9b6-43b2-a0b4-999f56ce322a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.335528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01519947-a9b6-43b2-a0b4-999f56ce322a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.338630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-01519947-a9b6-43b2-a0b4-999f56ce322a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:21.360629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:21.361737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:21.362654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:21.362654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:21.362654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:21.362654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:21.362654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 252/500] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:21.363243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8f77d-3dd1-4c20-8fc6-263cc7e948bf None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.363487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8f77d-3dd1-4c20-8fc6-263cc7e948bf None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.363764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8f77d-3dd1-4c20-8fc6-263cc7e948bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.381083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a8e8f77d-3dd1-4c20-8fc6-263cc7e948bf None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:21.382310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a8e8f77d-3dd1-4c20-8fc6-263cc7e948bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:21.384319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.459656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 253/501] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:21.460919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed1578-be53-4454-afda-b0a12d5f605f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.461205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d8ad1-dae1-4d90-80bd-15c7a9d9d757 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.461493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d8ad1-dae1-4d90-80bd-15c7a9d9d757 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.461772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d8ad1-dae1-4d90-80bd-15c7a9d9d757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.482987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-814d8ad1-dae1-4d90-80bd-15c7a9d9d757 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:21.484182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-814d8ad1-dae1-4d90-80bd-15c7a9d9d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:21.512675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.514134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.514453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.514729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.522381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.523842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:21.538091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:21.539005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:21.539864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 249/502] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:21.540424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.557569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 254/503] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:21.559268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d8ad1-dae1-4d90-80bd-15c7a9d9d757 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.619970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.645946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.673456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd1ad8-6ace-4242-aecd-123fd4cc8409 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.683176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d8ad1-dae1-4d90-80bd-15c7a9d9d757 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.710392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.711139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.711416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.711697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.712532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:21.712843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.714293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.719718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.720385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.720667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.720938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.721663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:21.721973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.723390 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.734793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 255/504] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:21.735621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.754255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 250/505] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:21.755115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.852285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81726e-83a6-4f96-8f53-e947f03d10ae None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.862035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7812e-d420-4c39-9102-a7f461041689 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:21.879727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.882419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:21.887296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.893606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.893606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.893606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.897505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:21.898211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:21.898541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:21.898825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:21.899671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:21.899985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.900115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:21.901438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.901541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:21.921038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:21.921938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:21.923696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:21.923696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:21.923696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:21.923696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:21.923986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 251/506] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:21.923986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:21.924132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 256/507] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:21.924132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.027689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812d200-8085-43a1-bd61-2e4a71ab8e0e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.028689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274499a-5aae-4160-a8cb-ffc7292ba9fc None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.061119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.061942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.062046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.062252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.062522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.062766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.063053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.063333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.072238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.073712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.075211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:22.083681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:22.084567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:22.085590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 252/508] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:22.086632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:22.090603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:22.091704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:22.092744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 257/509] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:22.093885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.127957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.140142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.186465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e58dd-fe6f-4bb0-992d-fa6f7e7769e5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.196358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd72a3-085c-47ac-a658-718c11329e89 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.224978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.225844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.225952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.226185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.231195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.231928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.237748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.237748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.293049 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.295221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.298984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.300464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.333641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.343775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.400353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 253/510] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:22.401471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.404931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 258/511] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:22.405921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.508547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfcae-d67c-47e1-91b8-1219173630a5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.510204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ebbd8-049c-4cf9-a227-d321cd6ffe88 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.546273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.547426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.547426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.547918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.548265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:22.548558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.550001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.551354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.552032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.552316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.552594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.553350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:22.553658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.555076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.557640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.563495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 254/512] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:22.563867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 259/513] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:22.564813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.568333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.568579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.675397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3621b-1c09-4729-94fd-c22a8c87012c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.682781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14591a-d0ac-4a77-b99f-5e8ee4ffeeed None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.720312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.720980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.721271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.721535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.722407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:22.722709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.724158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.727604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.728403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.728685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.728958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.742209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.744387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 255/514] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:22.744677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.744929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:22.767187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:22.768288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:22.773918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:22.773918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:22.773918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:22.773918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:22.773918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 260/515] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:22.773918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.815973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.835948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:22.866645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1d085-b2f0-4002-9a73-a6772c724ac2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.882102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebf87a-7c1d-4279-bd27-7c5b1e00adff None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:22.905070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.905872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.906152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.906449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.917958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.920629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.920740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:22.921439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:22.921725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:22.922005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:22.932949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:22.934383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:22.945592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:22.946491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:22.947324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:22.947324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:22.947324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:22.947324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:22.947324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 256/516] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:22.947968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:22.953114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:22.953970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:22.954818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:22.954818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:22.954818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:22.954818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:22.954818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 261/517] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:22.955441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.055296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09790de8-e696-4465-8333-fdcc33fdd6b8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.059296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940db7c-112a-4ab9-a38c-18a2cc6722c3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.065916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:23.067716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:23.093583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9c127-b969-414a-980e-3784db308805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.094297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c127-b969-414a-980e-3784db308805 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.094587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c127-b969-414a-980e-3784db308805 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.094866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c127-b969-414a-980e-3784db308805 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.095592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9c127-b969-414a-980e-3784db308805 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.097063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9c127-b969-414a-980e-3784db308805 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.098842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.099503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.099779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.100146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.171786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.173455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.189562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 257/518] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:35:22 2022] POST /identity/v3/users => generated 247 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:23.190423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9c127-b969-414a-980e-3784db308805 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.318608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 262/519] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:23.318608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.318868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9c127-b969-414a-980e-3784db308805 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.330490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:23.355924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.357150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.357440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.357725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.358730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:23.359043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.360449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.375885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 258/520] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:23.379245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.408925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:23.454013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91b04-7eb4-44fc-ab92-6d3dd2f92e74 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.496218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.496882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.497205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.497480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.498234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:23.498545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.499947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.516214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 263/521] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:23.516214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.521176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b59b4-6158-4b1d-ada9-a15d2184e1d1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.565580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.566273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.566555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.566829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.567558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:23.567868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.569244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.585629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 259/522] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:23.586846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.597236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:23.636606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfb87b-3fba-4ecf-b31b-c9cc68e6d111 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.693168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.694466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.694927 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.695382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.713545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.716523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.723152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:23.735350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:23.736384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:23.737295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:23.737295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:23.737295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:23.737295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:23.737295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 264/523] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:23.737582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.745214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f5532-a64f-4513-90a8-84d24fe4cbe5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.783467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088926d-e406-4745-83e5-3a442192392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.784271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088926d-e406-4745-83e5-3a442192392a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.784548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088926d-e406-4745-83e5-3a442192392a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.784820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088926d-e406-4745-83e5-3a442192392a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.792487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088926d-e406-4745-83e5-3a442192392a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.794522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088926d-e406-4745-83e5-3a442192392a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-3088926d-e406-4745-83e5-3a442192392a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:23.810934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:23.812117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 260/524] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:23.813269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088926d-e406-4745-83e5-3a442192392a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.848117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a50cc3-af6b-452e-a620-98a650c6d8b6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.869250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:23.892026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.895131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.895131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.895131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:23.901344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:23.903170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:23.919819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:23.921077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 265/525] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:23.927101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:23.948514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088926d-e406-4745-83e5-3a442192392a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:23.990454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:23.991184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:23.991472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:23.991751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.001713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.002750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:24.021301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:24.022298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:35:24.023232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:24.027370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 261/526] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:24.028376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.034471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:24.052541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6672b-7875-40d7-a487-e4b02260b6c7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.103824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.104617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.104896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.105176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.105926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:24.106257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.107798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.121034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 266/527] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:24.121891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.179725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:24.193107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d803d1-830a-46bf-9695-6d302c55afa0 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.238599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.239261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.239544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.239818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.240653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:24.240916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.242368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.252126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3dbc6-a010-4a78-9061-1e8358cec019 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.270368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 262/528] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:24.271919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.295654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.296338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.296620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.296893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.297684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:24.297995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.299574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.321601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 267/529] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:24.321601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.373256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:24.426950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a47ab-192b-4f77-bc40-b9a408d1243c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.451450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:24.455790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112386a-f545-47c5-ade6-ccb3c2970096 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.472795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.473604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.473897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.474395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.475543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:24.475863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.477357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.494050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.495725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.496007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.496284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.503776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 263/530] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:24.506295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.506425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.506622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:24.524030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:24.525165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:24.525996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 268/531] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:24.526474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.633574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf217f-78fc-4db8-a30a-8d0cf83cd0d7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.660403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:24.666731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01ca8d-716b-410a-9e4c-481c0ea2936d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.672958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.678334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.678334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.678334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.678334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=ac6a657099c14f598d5becd5ab72ee84) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:24.678334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.678334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.691974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 264/532] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/role_assignments?user.id=ac6a657099c14f598d5becd5ab72ee84&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:24.692534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.730113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.731234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.732312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.732747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.734145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=28faa0c6ac9b4e3a9528c3199506257c) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:24.734663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.737187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:24.745877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 269/533] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/role_assignments?user.id=28faa0c6ac9b4e3a9528c3199506257c&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:24.746727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:24.777983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:24.850403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa0b04-2771-458d-b0f1-8482cbe0ce53 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.908664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.909332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.909615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.909953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.941310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dff49-b007-4905-b96f-ef39c6be9502 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:24.992488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:24.993164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:24.993444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:24.993723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:24.994555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:24.994968 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:24.996513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:25.005804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:25.014732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 270/534] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:25.020970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:25.031494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:25.039375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:25.212779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 265/535] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:25.220156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:25.238075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:25.273121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd28aa-2200-4e56-8cfb-0d6915a84a02 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:25.416371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e86b3-ecaf-4bf9-a711-b38a18efb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:25.417035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e86b3-ecaf-4bf9-a711-b38a18efb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:25.417315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e86b3-ecaf-4bf9-a711-b38a18efb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:25.417589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e86b3-ecaf-4bf9-a711-b38a18efb4f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:25.418453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e86b3-ecaf-4bf9-a711-b38a18efb4f0 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=8d29198c768d4024af04273a3cca8f20) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:25.419288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e86b3-ecaf-4bf9-a711-b38a18efb4f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:25.421501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e86b3-ecaf-4bf9-a711-b38a18efb4f0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:25.423156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:25.441433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a283c-7ff4-4afa-982e-85314752cebb None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:25.445427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 271/536] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:25 2022] GET /identity/v3/role_assignments?user.id=8d29198c768d4024af04273a3cca8f20&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 259 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:25.486829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:25.487677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:25.489186 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:25.489504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:25.517733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:25.520780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:25.541436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe912f-aff9-4aa6-97d1-b7187190197c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:25.541730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe912f-aff9-4aa6-97d1-b7187190197c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:25.541994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe912f-aff9-4aa6-97d1-b7187190197c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:25.543704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:25.545709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:25.546755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:25.546755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:25.546755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:25.546755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:25.547007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 272/537] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:25.547086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 266/538] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:25.554210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10ba4-14c1-40bf-b876-31d6c63644cd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:25.554210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10ba4-14c1-40bf-b876-31d6c63644cd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:25.554210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10ba4-14c1-40bf-b876-31d6c63644cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:25.557491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:25.595236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-43f10ba4-14c1-40bf-b876-31d6c63644cd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:25.596793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-43f10ba4-14c1-40bf-b876-31d6c63644cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:25.736625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 273/539] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:25.760537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10ba4-14c1-40bf-b876-31d6c63644cd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:25.767813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129e519-a648-4adf-a299-701afb2691f3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:25.820796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:25.821412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:25.821696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:25.821972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:25.822896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:25.823482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:25.825446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:25.832718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:25.839437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 267/540] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:25.845470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:25.856048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:25.881649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10ba4-14c1-40bf-b876-31d6c63644cd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:25.922309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:25.923223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:25.923740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:25.924095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:25.936362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:25.944776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:25.947394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:25.949314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 274/541] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:25.958194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:25.958452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3111f66-ae31-415b-95f1-679debcacce2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:25.991691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:25.992493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:25.992873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:25.993175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.001556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.003046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:26.018323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:26.020869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:26.022895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 268/542] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:26.030199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:26.068806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928023e-cac1-404d-9079-bd9fb75a4173 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:26.089023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:26.112124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:26.112872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.113160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.113441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.114780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:26.115745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.118063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.120854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:26.130315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 275/543] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:26.135954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:26.137366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce500ad-5e48-447b-b52e-dccb57bfc3f5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:26.173375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:26.174476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.174907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.175330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.176270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:26.176726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.178362 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.192347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 269/544] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:26.198398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:26.294222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0afc0-3bf7-423f-979c-a8d5eb67e272 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:26.333241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b76c1-f934-40e3-a60b-65a37dc3c5a0 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:26.347192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:26.347918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.348947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.348947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.369116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.370662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.390601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:26.404530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:26.405600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:26.423149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:26.423149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:26.423149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:26.423149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:26.423149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 276/545] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:26.423516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:26.423516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.423516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.423516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.423819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=bead604b09be482c906a97b53bb03d69) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:26.423903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:26.423982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.423982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.427833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 270/546] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:26 2022] GET /identity/v3/role_assignments?user.id=bead604b09be482c906a97b53bb03d69&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:26.469485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:26.528917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa93b-fcb6-4550-bd1e-b77a94e05c51 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:26.568413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:26.569138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.569423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.569752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.570520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:26.570833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.572461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.586072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 277/547] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:26.595928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:26.814706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2f7d6-cd58-4ae7-b5cd-964e01a8f8f6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:26.863885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:26.864708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:26.864988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:26.865263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] PATH_INFO: `/v3/users/swift` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:26.879426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:26.880968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:26.899118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:26.900258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:26.901372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 271/548] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:26.910477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:27.092624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09defa-0e0a-4145-87e0-f2ac34967692 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:27.105255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:27.133433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540565cf-621b-4782-90cc-7672ba29ff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:27.134397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540565cf-621b-4782-90cc-7672ba29ff19 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:27.134785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540565cf-621b-4782-90cc-7672ba29ff19 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:27.135220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540565cf-621b-4782-90cc-7672ba29ff19 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:27.136415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540565cf-621b-4782-90cc-7672ba29ff19 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:27.136839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540565cf-621b-4782-90cc-7672ba29ff19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:27.139635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540565cf-621b-4782-90cc-7672ba29ff19 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:27.193777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 278/549] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Jun 15 14:35:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:27.200770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:27.260981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:27.359108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957195d-c9c6-4039-8520-275c587a0cdd None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:27.412387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:27.413078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:27.413361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:27.413637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:27.500334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de73106-819e-4d4c-8efc-cd3afa9f0914 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:27.500673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de73106-819e-4d4c-8efc-cd3afa9f0914 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:27.500897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de73106-819e-4d4c-8efc-cd3afa9f0914 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:27.502280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 279/550] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:27.507690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb352-ff15-4e3c-a6e4-fd8bf453ed80 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:27.507979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb352-ff15-4e3c-a6e4-fd8bf453ed80 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:27.508259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb352-ff15-4e3c-a6e4-fd8bf453ed80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:27.516978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:27.518547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:27.547275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-2aebb352-ff15-4e3c-a6e4-fd8bf453ed80 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:27.548514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2aebb352-ff15-4e3c-a6e4-fd8bf453ed80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:27.570542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:27.704740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 272/551] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:27.706793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 280/552] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:27.708624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:27.736200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb352-ff15-4e3c-a6e4-fd8bf453ed80 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:27.920316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa3fc2-d660-455e-9e68-19d3ee9acee9 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:27.978078 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:27.985074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb352-ff15-4e3c-a6e4-fd8bf453ed80 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.006968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108e1ec-23f5-447a-8781-38a585222a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.007780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108e1ec-23f5-447a-8781-38a585222a0f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.008063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108e1ec-23f5-447a-8781-38a585222a0f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.008338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108e1ec-23f5-447a-8781-38a585222a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.031956 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108e1ec-23f5-447a-8781-38a585222a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.033670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108e1ec-23f5-447a-8781-38a585222a0f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.038476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.052919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1e89d-7c9e-4940-83a8-0372e38bf9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.053929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e89d-7c9e-4940-83a8-0372e38bf9e1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.054025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e89d-7c9e-4940-83a8-0372e38bf9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.054270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e89d-7c9e-4940-83a8-0372e38bf9e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-4108e1ec-23f5-447a-8781-38a585222a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:28.055902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:28.056831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:28.057676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:28.057676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:28.057676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:28.057676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:28.060500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 273/553] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:28.062027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef4ea6-a9e5-467b-96fa-18372b4032e5 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.062545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef4ea6-a9e5-467b-96fa-18372b4032e5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.063040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef4ea6-a9e5-467b-96fa-18372b4032e5 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.065157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 274/554] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:28.065263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1e89d-7c9e-4940-83a8-0372e38bf9e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:28.066713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2259-d01b-41af-950f-646490a4eac9 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.067195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2259-d01b-41af-950f-646490a4eac9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.067693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2259-d01b-41af-950f-646490a4eac9 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.073345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 275/555] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-cdb1e89d-7c9e-4940-83a8-0372e38bf9e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:28.073688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:28.074591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:28.075839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:28.078056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2259-d01b-41af-950f-646490a4eac9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.080015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 281/556] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:28.081406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352f9ff-7ef9-42e7-bd67-42160a38e592 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.081696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352f9ff-7ef9-42e7-bd67-42160a38e592 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.081975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352f9ff-7ef9-42e7-bd67-42160a38e592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.099500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6352f9ff-7ef9-42e7-bd67-42160a38e592 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:28.100695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6352f9ff-7ef9-42e7-bd67-42160a38e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:28.179325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 282/557] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:28.180749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fce191-2cf0-43ce-a745-28e8bbc4158f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.181035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fce191-2cf0-43ce-a745-28e8bbc4158f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.181307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fce191-2cf0-43ce-a745-28e8bbc4158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.203588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-59fce191-2cf0-43ce-a745-28e8bbc4158f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:28.204942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-59fce191-2cf0-43ce-a745-28e8bbc4158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:28.235182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2259-d01b-41af-950f-646490a4eac9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.269278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.270192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.273380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.273380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.273380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:28.273380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.273380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.286378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 283/558] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:28.286378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fce191-2cf0-43ce-a745-28e8bbc4158f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.292935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 276/559] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:28.292935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.310863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.364942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.395777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fce191-2cf0-43ce-a745-28e8bbc4158f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.412801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e77a8a-31d1-4a65-a440-6970bc5ae56a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.431699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.432364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.432644 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.432917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.434081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:28.434497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.436079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.444261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 284/560] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:28.445054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.457417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.458241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.458519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.459182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.470991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:28.478612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:28.479551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 277/561] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:28.481864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.537748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.562422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c0721-0207-4050-9995-6045bb5896d0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.596936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c8b5-8e1e-4258-9058-6a525e4026f6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.607705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.608514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.608813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.609469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.613658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.616510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.617981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:28.633696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:28.634726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:28.635630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 285/562] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:28.636184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.641157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.641668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.642399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.643097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.654674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:28.662490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:28.663483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:28.664284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 278/563] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:28.664954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.752011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a97a4c-9446-4daa-87f2-ab83d51bca25 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.773660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830ed3-05a2-47ec-a303-e1bf8628fa07 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.789221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.789944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.790259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.790543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.799977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.802568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.803864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.811036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.811803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.812012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.812261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.818876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:28.818876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.818876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:28.822448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:28.823311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:28.824826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:28.824826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:28.824826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:28.824826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:28.825219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 279/564] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:28.825219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.833478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 286/565] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:28.833478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:28.866028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:28.941636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252fb4e-7286-40eb-b61c-69460b1d62bf None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.945584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a16863-13c4-4a09-9f9f-836ee8ca127f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:28.978309 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.978993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.979278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.979558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.980355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:28.980708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.982954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.983944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:28.984906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:28.985356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:28.985817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:28.986726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:28.987222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:28.988754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:28.994787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 280/566] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:28.995597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.002012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 287/567] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:29.003114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.064086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.095356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.101833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadc56-8f9d-4adc-8722-4df201787410 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.118765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d722ebe-3861-44c5-803d-db29978730d6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.141113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.142405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.143195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.143480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.154699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.156127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.157046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.157718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.158008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.158315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.159050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:29.159357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.160746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.173588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 288/568] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:29.174439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:29.175666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:29.176538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:29.178298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:29.178298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:29.178298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:29.178298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:29.178298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 281/569] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:29.178298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.281243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643d70b-e116-4a5c-b8b1-e0b9d4c52443 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.284439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed499-16d6-4c8d-8b05-ca17de02143a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.295870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.315552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a97a4a-9d97-4122-b4a2-fb04094ea0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.316178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a97a4a-9d97-4122-b4a2-fb04094ea0b5 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.316845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a97a4a-9d97-4122-b4a2-fb04094ea0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.317127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a97a4a-9d97-4122-b4a2-fb04094ea0b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.317963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a97a4a-9d97-4122-b4a2-fb04094ea0b5 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=0b3a2fb853764be199cc6ff0dfd18064) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:29.318302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a97a4a-9d97-4122-b4a2-fb04094ea0b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.319757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a97a4a-9d97-4122-b4a2-fb04094ea0b5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.338199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 289/570] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/role_assignments?user.id=0b3a2fb853764be199cc6ff0dfd18064&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:29.338199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bed7b0-f990-4a27-85a5-9807e96f8048 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.338199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bed7b0-f990-4a27-85a5-9807e96f8048 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.338199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bed7b0-f990-4a27-85a5-9807e96f8048 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.338199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 290/571] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:29.338199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bed7b0-f990-4a27-85a5-9807e96f8048 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.338798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.338798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.338989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.339294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.352415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.353916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.364651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:29.375852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:29.382967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:29.384132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:29.384132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:29.384132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:29.384132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:29.384132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 282/572] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:29.384132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.519605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d07be1-738f-4189-a0d2-ca52767c5056 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.534218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bed7b0-f990-4a27-85a5-9807e96f8048 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.568244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1c65b-51b6-44c1-8d1a-a39e9f87eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.568810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1c65b-51b6-44c1-8d1a-a39e9f87eb5c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.569092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1c65b-51b6-44c1-8d1a-a39e9f87eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.569352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1c65b-51b6-44c1-8d1a-a39e9f87eb5c None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.570086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1c65b-51b6-44c1-8d1a-a39e9f87eb5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:29.570438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1c65b-51b6-44c1-8d1a-a39e9f87eb5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.571850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1c65b-51b6-44c1-8d1a-a39e9f87eb5c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.586777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 283/573] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:29.588366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359d3d1-c10b-4eda-aeaa-5688d577de41 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.588660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359d3d1-c10b-4eda-aeaa-5688d577de41 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.589030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359d3d1-c10b-4eda-aeaa-5688d577de41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.597897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.599636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.600084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.600509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] PATH_INFO: `/v3/users/glance` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.613908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.616093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.618364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3359d3d1-c10b-4eda-aeaa-5688d577de41 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:29.620422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3359d3d1-c10b-4eda-aeaa-5688d577de41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:29.642930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:29.645320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:35:29.647911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:29.652108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 291/574] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:29.653101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.702500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.707290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.720501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 284/575] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:29.721380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359d3d1-c10b-4eda-aeaa-5688d577de41 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.788795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f164-e148-413e-af39-bb16cbcb4b0c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.842868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.843453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.843740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.844016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.844866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:29.845180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.846768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.850369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359d3d1-c10b-4eda-aeaa-5688d577de41 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:29.865254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 292/576] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:29.865254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.911445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:29.912511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:29.912960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:29.913237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:29.923190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:29.924699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:29.941776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:29.942821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:29.943731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:29.947817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 285/577] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:29.948607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:29.992589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:29.999085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2f66-9c3f-48c7-849e-c374c67a9339 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.004547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:30.038847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.039531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.039821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.040098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.041003 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:30.041350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.042899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.060606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbfa4f-c78e-4777-9872-619ce1afe870 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.066742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 293/578] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:30.067449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.101584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.102885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.103346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.103839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.123240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:30.138640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:30.139789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:30.140660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 286/579] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:30.141345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.195024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed02625-42f3-49bd-8959-ca35fcb86f0a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.239744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.240699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.240976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.241239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.249931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.251456 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.252555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882aa9-217a-4189-96fb-b810f225782b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.264114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:30.267322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:30.268243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:30.269144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 294/580] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:30.269807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.290115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:30.295985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.297227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.297866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.298144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.299033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:30.299733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.301570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.322548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 287/581] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:30.323849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.394535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1de4c3-4025-475e-872c-0ef499658e23 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.432468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.433272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.433560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.433844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.444230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47a45f-4cc9-40d5-8983-3e682be59f90 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.448143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.450021 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:30.470358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:30.471319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:30.472392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:30.472392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:30.472392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:30.472392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:30.474005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 295/582] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:30.474859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.510771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.511689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.512603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.512947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.513828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:30.514277 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.515885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.531743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 288/583] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:30.532563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.572555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:30.589825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:30.628253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51523878-5750-4bbb-96d5-38be29242c04 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.664585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4df597-e3fd-4431-a550-15d62601a64c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.705245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.706309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.706684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.707096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.708156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:30.708603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.710946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.717231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.718046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.718367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.718646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.731210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.736877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 296/584] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:30.737039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.737132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:30.751964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:30.753131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:30.754215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:30.754215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:30.754215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:30.754215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:30.754215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 289/585] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:30.754568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.847753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:30.865124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2f90-6957-4e0d-88d6-83a05e2fff4f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.875583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092607a-f3d1-4ed2-966e-719eab461650 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:30.894635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:30.904380 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.905194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.905584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.906027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.906900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:30.907328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.908747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.921271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:30.922334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:30.922768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:30.927272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 297/586] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:30.927272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:30.927912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:30.944490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:30.946511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:30.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:30.977526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:30.979368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:30.979368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:30.979368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:30.979368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:30.979835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 290/587] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:30.980731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.055721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aaf83-0019-4bff-8223-85d0d9e7095f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.099181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.099985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.100272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.100625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.124259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.125502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b690c4-d5b6-41ae-bcb2-373aab24e94e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.125755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:31.145060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:31.146011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:31.146915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:31.146915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:31.146915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:31.146915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:31.150834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 298/588] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:31.153466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.171773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.172353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.172651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.172906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.173857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:31.174145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.175785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.177922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:31.191529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 291/589] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:31.196690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.229196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:31.313868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cdb73-e1d0-450f-be9a-6ac61dda01c8 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.340420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf603-99c2-4e79-8fba-2d9e22d29492 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.357748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.358590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.358875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.359144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.366754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.368219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.382160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.383113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.383496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.383876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.384715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:31.385128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:31.385801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:31.388218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:31.389207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 299/590] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:31.389896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.392470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.406076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 292/591] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:31.407292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.486042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:31.514305 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:31.519197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e352bf-3eea-4833-8f41-eb03cab4d23d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.541661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f1e5-fddb-43f7-95b1-1c5e7c201307 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.565215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.565894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.566206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.566502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.567248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:31.567560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.568966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.583001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 300/592] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:31.583661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.588717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.589523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.589800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.590142 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.599129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.600789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:31.616174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:31.617626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:31.619295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:31.622828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 293/593] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:31.623645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.714598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3dfcb-efa8-4be8-a96a-af345b93c42c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.745093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244541b5-fed7-47ab-b748-7dca9ff4dbdb None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.758992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.759994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.760468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.760881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.762662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:31.763156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.765144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.767619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:31.779997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 301/594] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:31.780855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.796513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.797320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.797606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.797882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.802679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:31.805335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.806862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:31.822792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:31.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 294/595] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:31.830508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:31.901622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270936d1-78ac-44b7-a83b-a9ca68e2a384 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.943721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da733f8-33b5-41c7-8135-0c0975b2c030 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:31.979772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375f7ae-3852-456c-bc0a-8e0aa13c2102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.980678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375f7ae-3852-456c-bc0a-8e0aa13c2102 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.980937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375f7ae-3852-456c-bc0a-8e0aa13c2102 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.981229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375f7ae-3852-456c-bc0a-8e0aa13c2102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.985882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:31.986925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:31.987382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:31.987815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:31.988572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375f7ae-3852-456c-bc0a-8e0aa13c2102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.989098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:31.989599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:31.990049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375f7ae-3852-456c-bc0a-8e0aa13c2102 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:31.992113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-7375f7ae-3852-456c-bc0a-8e0aa13c2102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:32.004717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:32.005870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:32.007192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 302/596] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:32.007952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbeeea-047b-4365-81e3-948f37bcd963 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.007952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbeeea-047b-4365-81e3-948f37bcd963 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.008142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbeeea-047b-4365-81e3-948f37bcd963 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.010548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 303/597] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:32.010548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbeeea-047b-4365-81e3-948f37bcd963 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.015126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 295/598] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:32.016286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.058329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:32.123418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:32.132805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1ae0-e7ec-481a-a105-99c6562ad1f3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.145071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbeeea-047b-4365-81e3-948f37bcd963 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.166374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.167041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.167317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.167588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.169421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:32.169735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:32.171179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.179874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca78cb-ace5-4a5a-96d4-63520979b940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.180765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca78cb-ace5-4a5a-96d4-63520979b940 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.181042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca78cb-ace5-4a5a-96d4-63520979b940 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.181314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca78cb-ace5-4a5a-96d4-63520979b940 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/28faa0c6ac9b4e3a9528c3199506257c/roles/44450886f4e74405a0d654944c8740d4` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.189058 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 296/599] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Jun 15 14:35:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:32.189861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.197056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca78cb-ace5-4a5a-96d4-63520979b940 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=28faa0c6ac9b4e3a9528c3199506257c, role_id=44450886f4e74405a0d654944c8740d4)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:32.198543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca78cb-ace5-4a5a-96d4-63520979b940 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.258480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 304/600] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:32 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/28faa0c6ac9b4e3a9528c3199506257c/roles/44450886f4e74405a0d654944c8740d4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:32.259687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22c740-79dc-4320-a37e-26aeb5c54630 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.259977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22c740-79dc-4320-a37e-26aeb5c54630 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.260423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22c740-79dc-4320-a37e-26aeb5c54630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.284464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:32.294374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d7de9-6005-4526-a5d7-0fff3207752c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.299818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7a22c740-79dc-4320-a37e-26aeb5c54630 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:32.301217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7a22c740-79dc-4320-a37e-26aeb5c54630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:32.345131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.346138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.346620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.347042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.348335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:32.348802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:32.351701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.375082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 297/601] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:32.375082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.394096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 305/602] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:32.395345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22c740-79dc-4320-a37e-26aeb5c54630 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.463850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:32.510437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22c740-79dc-4320-a37e-26aeb5c54630 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.510621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f8ef9-ddca-46f8-8c99-ceccc61d3544 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.548078 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.548931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.549228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.549521 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.555313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.556516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.557005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.557494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/ac6a657099c14f598d5becd5ab72ee84/roles/44450886f4e74405a0d654944c8740d4` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.563126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:32.563979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.576798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=ac6a657099c14f598d5becd5ab72ee84, role_id=44450886f4e74405a0d654944c8740d4)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:32.578602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:32.582124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:32.583027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:32.583914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:32.583914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:32.583914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:32.583914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:32.583914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 306/603] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:32.584192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.671332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 298/604] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:32 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/ac6a657099c14f598d5becd5ab72ee84/roles/44450886f4e74405a0d654944c8740d4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:32.672656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.686230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:32.707398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727303e3-e9ca-4f00-a1e1-cfec4d1da71e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.729197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:32.773720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.775244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.778529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.779053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/8d29198c768d4024af04273a3cca8f20/roles/44450886f4e74405a0d654944c8740d4` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.804981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=8d29198c768d4024af04273a3cca8f20, role_id=44450886f4e74405a0d654944c8740d4)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:32.808579 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:32.813161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7346703-7755-4671-a0cc-79659f461422 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:32.852948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:32.853922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:32.854344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:32.854720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:32.869375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:32.877872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:32.878858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:32.879677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 299/605] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:32.880364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.887908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 307/606] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:32 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/8d29198c768d4024af04273a3cca8f20/roles/44450886f4e74405a0d654944c8740d4 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:32.888747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:32.983911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:33.039962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89057-aab0-4a8c-ad39-f9ec6ae7ec36 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.055914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13573-613e-4102-8141-91ef1cccdf10 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.093646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.096644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.097190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.097692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.098897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:33.100630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.104311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:33.123045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 300/607] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:33.129303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:33.133207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.133894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.134199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.134479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.135244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:33.135553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.137493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:33.147630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 308/608] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:33.155675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:33.194533 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:33.258351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0033-73c2-4519-b935-aab2f2a1ce49 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.285013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:33.307963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.308925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.309230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.309509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.320699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.322160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:33.336410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25b56f-e2c4-4e92-a004-545d583d7c1e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:33.348698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:33.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:33.351613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 301/609] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:33.356468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:33.399807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.400954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.401238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.401514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.434679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.437029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:33.474807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:33.476426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:33.477574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:33.477574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:33.477574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:33.477574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:33.482654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 309/610] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:33.496716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:33.531361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f9e5a-4aed-46f0-b3d5-d0f149d426e0 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.590342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.591489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.591993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.592483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.593952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:33.594485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.596128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:33.634090 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 302/611] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:33.642846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:33.698679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:33.739534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:33.743214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef439f-3b68-41ac-a81b-cf3040f5f180 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.809875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.810740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.811195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.811598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.812340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:33.812647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.814423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:33.831286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 310/612] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:33.836875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bc63c-b47b-4fc2-b41f-0ad4d0fb5ef7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.837224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:33.915857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:33.917171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:33.917457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:33.917733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/bead604b09be482c906a97b53bb03d69/roles/44450886f4e74405a0d654944c8740d4` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:33.967618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=bead604b09be482c906a97b53bb03d69, role_id=44450886f4e74405a0d654944c8740d4)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:33.969000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef900e-c789-4953-bbde-e80aa8aab8ce None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:33.969161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:34.012937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:34.012937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:34.013238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:34.013455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:34.023518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:34.025393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:34.034699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:34.044371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:34.045376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:34.047597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 311/613] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:34.051989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:34.077112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 303/614] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:33 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/bead604b09be482c906a97b53bb03d69/roles/44450886f4e74405a0d654944c8740d4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:35:34.196869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76238b-8b24-4714-93ab-56f6c41aa2d0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:34.240927 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:34.242099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:34.242774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:34.243052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:34.244047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:34.244359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:34.245841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:34.271271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 312/615] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Jun 15 14:35:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:34.277124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:34.426110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:34.439775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195f312-324b-4f89-b5ee-b84b7e16f468 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:34.486124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:34.486890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:34.487175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:34.487453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:34.501935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:34.503532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:34.526673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:34.527839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:34.528798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:34.528798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:34.528798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:34.528798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:34.528798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 304/616] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:34.536306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:34.656623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:34.720760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aa69a-e514-406b-91c9-4b8e1fbf7a15 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:34.766719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:34.767748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:34.768236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:34.768718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:34.769641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:34.770155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:34.772053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:34.786254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 313/617] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:34.791358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:34.948558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a89a-ff41-4c6f-b3af-2982283b49f6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:35.001682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedd0ef-b48e-4108-8f17-14cd80173954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:35.002516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd0ef-b48e-4108-8f17-14cd80173954 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.002798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd0ef-b48e-4108-8f17-14cd80173954 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.003071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd0ef-b48e-4108-8f17-14cd80173954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.010397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedd0ef-b48e-4108-8f17-14cd80173954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:35.012085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedd0ef-b48e-4108-8f17-14cd80173954 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-0dedd0ef-b48e-4108-8f17-14cd80173954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:35.034609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:35.035547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:35.036417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 305/618] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:35.041633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:35.202264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab36bd-4e64-4056-b587-e8d0b58a436d None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.202635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab36bd-4e64-4056-b587-e8d0b58a436d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.202829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab36bd-4e64-4056-b587-e8d0b58a436d None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.205320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 306/619] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:35.211940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd257c-111d-485a-8511-2ad2b8202165 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.212237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd257c-111d-485a-8511-2ad2b8202165 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.212518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd257c-111d-485a-8511-2ad2b8202165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.214190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afd6e9-2f6a-475d-8708-99f5a74d679e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:35.229662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:35.231418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:35.247179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-babd257c-111d-485a-8511-2ad2b8202165 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:35.248434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-babd257c-111d-485a-8511-2ad2b8202165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:35.253558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:35.254525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.254984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.255434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.256674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:35.257269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:35.258980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:35.274775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 314/620] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:35.282074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:35.390413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 307/621] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:35.418354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd257c-111d-485a-8511-2ad2b8202165 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:35.477657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311540f-a12d-4890-bf48-1da10ba43d7f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:35.526439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:35.527887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.528153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.528438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/0b3a2fb853764be199cc6ff0dfd18064/roles/44450886f4e74405a0d654944c8740d4` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.558197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=0b3a2fb853764be199cc6ff0dfd18064, role_id=44450886f4e74405a0d654944c8740d4)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:35.558197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:35.602371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd257c-111d-485a-8511-2ad2b8202165 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:35.644078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f81a5-808b-4839-9024-71b8ac2997de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:35.644872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f81a5-808b-4839-9024-71b8ac2997de None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.645161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f81a5-808b-4839-9024-71b8ac2997de None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.645485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f81a5-808b-4839-9024-71b8ac2997de None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.646515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 315/622] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:35 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/0b3a2fb853764be199cc6ff0dfd18064/roles/44450886f4e74405a0d654944c8740d4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:35.648058 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:35.659911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f81a5-808b-4839-9024-71b8ac2997de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-a33f81a5-808b-4839-9024-71b8ac2997de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:35.670508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:35.671498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:35.672341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:35.673600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 308/623] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:35.679405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:35.711594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:35.745263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b81b3-53e6-42ac-b8ba-3d87800b60b1 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.745659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b81b3-53e6-42ac-b8ba-3d87800b60b1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.746567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b81b3-53e6-42ac-b8ba-3d87800b60b1 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.748393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 309/624] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:35.755128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4ee0-2db2-4b49-9e64-b7fddfe37771 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.755402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4ee0-2db2-4b49-9e64-b7fddfe37771 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.755764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4ee0-2db2-4b49-9e64-b7fddfe37771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.787863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-49df4ee0-2db2-4b49-9e64-b7fddfe37771 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:35.790883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-49df4ee0-2db2-4b49-9e64-b7fddfe37771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:35.868051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17314021-bdf4-44c6-9e90-1d2bae7898cb None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:35.903203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 310/625] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:35.904223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20647842-abdc-4465-9870-ea54ea9cc378 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.904515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20647842-abdc-4465-9870-ea54ea9cc378 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.904798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20647842-abdc-4465-9870-ea54ea9cc378 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.906462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 311/626] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:35.917112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:35.918374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.918658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.918931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.919880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:35.920169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:35.921565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:35.925188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fd67e-46e1-4986-a955-3f45dadc3987 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:35.925481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fd67e-46e1-4986-a955-3f45dadc3987 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:35.925766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fd67e-46e1-4986-a955-3f45dadc3987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:35.930950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 316/627] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:35.937255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:35.953453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-9f2fd67e-46e1-4986-a955-3f45dadc3987 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:35.954684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9f2fd67e-46e1-4986-a955-3f45dadc3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:36.051418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:36.058844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 312/628] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:36.060635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fd67e-46e1-4986-a955-3f45dadc3987 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.081606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f6cf8-43de-4b7b-a7ca-9310d9426bc8 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.131887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.132883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.133267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.133546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.147541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:36.148366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:36.160913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:36.162428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:36.163562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 317/629] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:36.167302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.213879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fd67e-46e1-4986-a955-3f45dadc3987 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.268665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.269825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.270353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.270849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.282523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:36.284296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:36.303549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:36.304593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:36.305456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:36.305456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:36.305456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:36.305456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:36.307767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 313/630] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:36.309506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.310284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488fd15-5bc5-404b-ac19-0cf0a28a5283 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.356900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.358736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.359243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.359715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.373176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:36.383089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:36.384037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:36.384866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:36.388267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 318/631] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:36.389356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.399679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:36.448666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2837d-d4ba-4474-8593-14ad54cf86db None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.458198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:36.498333 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.499015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.499303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.499584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.500346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:36.500659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:36.502132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:36.518506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 314/632] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:36.518506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.545157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06dbb2b-741e-4560-ad8d-a4c6c395d4a8 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.593848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.595582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.596639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.597128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.598642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:36.599429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:36.602013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:36.619351 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 319/633] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:36.620540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.638451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05450-19d3-4747-bed1-7520d12cbc7a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.675993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.676674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.676961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.677241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.677995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:36.678335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:36.679721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:36.691211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 315/634] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:36.696395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.712286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:36.741468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:36.766532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a25a-a85f-4527-b639-42c3f74d2cc0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.813560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.815027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.815316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.815594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.822803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b551bd-b259-4ca3-b7bc-65db4cb7c5d9 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:36.828471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:36.829962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:36.849320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:36.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:36.851192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:36.851192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:36.851192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:36.851192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:36.855485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 320/635] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:36.856596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.876329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:36.878029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:36.878660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:36.879711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:36.890987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:36.892482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:36.909520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:36.910615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 14:35:36.911558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:36.915597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 316/636] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:36.916757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:36.989188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a7779-25b7-4293-b6da-dd5f3bde6586 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.007709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.021387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.028988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.030059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.030578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.031009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.031300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96c51-f22d-488e-90de-52baa10d8901 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.042904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.047239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:37.062906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:37.063825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:37.064792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:37.064792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:37.064792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:37.064792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:37.064792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 321/637] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:37.065073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.075016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a7ac4-0055-490e-8d42-1fdf626cfba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.075958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a7ac4-0055-490e-8d42-1fdf626cfba0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.076426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a7ac4-0055-490e-8d42-1fdf626cfba0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.076886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a7ac4-0055-490e-8d42-1fdf626cfba0 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.078322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a7ac4-0055-490e-8d42-1fdf626cfba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:37.078822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a7ac4-0055-490e-8d42-1fdf626cfba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.080868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a7ac4-0055-490e-8d42-1fdf626cfba0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.101359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 317/638] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:37.101359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b185-00b3-4617-8a93-175803cd3363 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.101359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b185-00b3-4617-8a93-175803cd3363 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.101359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b185-00b3-4617-8a93-175803cd3363 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.101735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 318/639] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:37.102304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0b185-00b3-4617-8a93-175803cd3363 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.181778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77a613-e3df-4358-bcbe-7a9fa097b172 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.224124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0b185-00b3-4617-8a93-175803cd3363 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.225170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.225844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.226125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.226474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.230251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:37.230251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.230251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.250028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 322/640] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:37.250938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.268654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d6ebb-c22b-4452-a7b7-02d310c979dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.269317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6ebb-c22b-4452-a7b7-02d310c979dd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.269603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6ebb-c22b-4452-a7b7-02d310c979dd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.269875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6ebb-c22b-4452-a7b7-02d310c979dd None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.270635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d6ebb-c22b-4452-a7b7-02d310c979dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:37.270937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d6ebb-c22b-4452-a7b7-02d310c979dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.272304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d6ebb-c22b-4452-a7b7-02d310c979dd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.283926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.288466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 319/641] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:37.289691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5406-fb54-41ea-90d4-c8ad84da603e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.289978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5406-fb54-41ea-90d4-c8ad84da603e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.290280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5406-fb54-41ea-90d4-c8ad84da603e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.307326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-686e5406-fb54-41ea-90d4-c8ad84da603e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:37.309150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-686e5406-fb54-41ea-90d4-c8ad84da603e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:37.311941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.365070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fa5d3-6904-49d4-be0b-eee3370ec77b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.400447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 320/642] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:37.402281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5406-fb54-41ea-90d4-c8ad84da603e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.415154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.416517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.417014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.417505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.432750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.436097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:37.459671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:37.460742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 323/643] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:37.462406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.542410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5406-fb54-41ea-90d4-c8ad84da603e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.567591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.585209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5289ee-9307-492d-a4b6-55b9af62d3b9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.585513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.586228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.586495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.586761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.599537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.625776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.626631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.626919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.627199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.636108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.638458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:37.654982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:37.655991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 324/644] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:37.657065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.671843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.673348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:37.774597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff7dcc-453b-4eef-ab0c-187c69479d98 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.805222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 321/645] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:37.805222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.823470 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.824387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.824668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.824962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.841868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:37.849628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:37.850631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:37.851467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 325/646] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:37.853278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:37.858330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.862227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:37.926586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f5e6f-1161-476d-952b-04b6a4d0dd75 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.977700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab1c9-7698-47d0-aef1-d88a8fef9b3a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:37.984477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:37.985197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:37.985522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:37.985841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:37.986804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:37.987222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:37.989780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.015846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 322/647] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:38.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.026357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.027702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.028483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.028483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.029137 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:38.029447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.030957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.049232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 326/648] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Jun 15 14:35:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:38.050069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.110870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.120119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.127479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7f08-eed3-4a03-9bba-62448230805c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.147477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ddf3f-da06-4c31-862b-0ee215903a85 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.163875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.164672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.164953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.165225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.175570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.177128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.182721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05becbd1-998f-4c83-a6d4-ae7385f0c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.183396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05becbd1-998f-4c83-a6d4-ae7385f0c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.183733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05becbd1-998f-4c83-a6d4-ae7385f0c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.184117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05becbd1-998f-4c83-a6d4-ae7385f0c9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.185046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05becbd1-998f-4c83-a6d4-ae7385f0c9d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:38.185502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05becbd1-998f-4c83-a6d4-ae7385f0c9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.187676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05becbd1-998f-4c83-a6d4-ae7385f0c9d3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:38.195650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:38.196565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:38.197443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:38.197443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:38.197443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.197443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:38.197668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 327/649] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:38.197740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 323/650] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:38.197815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e343a-f045-465c-b659-0644761d98ce None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.197815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e343a-f045-465c-b659-0644761d98ce None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.198417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e343a-f045-465c-b659-0644761d98ce None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.204185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 328/651] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:38.204185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e343a-f045-465c-b659-0644761d98ce None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.204697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.306957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcfb5a-9cae-41d2-820f-384d6411b3a9 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.312468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e343a-f045-465c-b659-0644761d98ce None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.339308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.342606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.343261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.343542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.343816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.348275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.348883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.349172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.349449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.363313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.415532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.416853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.425201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.426722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.520281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 324/652] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:38.521534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.536885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 329/653] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:38.536885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.564490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.575656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.630644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af6ea-5529-4a14-b2cf-e829176e0066 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.640749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec0a1c-66e9-4c0f-abd1-853fb6eca905 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.666639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.667437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.668285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.668563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.676271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3015d-f135-4b3d-8a2e-c4c9ea572ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.676930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3015d-f135-4b3d-8a2e-c4c9ea572ae8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.677212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3015d-f135-4b3d-8a2e-c4c9ea572ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.677487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3015d-f135-4b3d-8a2e-c4c9ea572ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.678226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3015d-f135-4b3d-8a2e-c4c9ea572ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:38.678536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3015d-f135-4b3d-8a2e-c4c9ea572ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.679458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.679919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3015d-f135-4b3d-8a2e-c4c9ea572ae8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.680894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.691213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 330/654] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:38.692611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9350e-4c18-4af9-85bd-046508ab5b20 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.692900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9350e-4c18-4af9-85bd-046508ab5b20 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.693163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9350e-4c18-4af9-85bd-046508ab5b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:38.698587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:38.700434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:38.701315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:38.701315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:38.701315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.701315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:38.701315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 325/655] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:38.701315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.710477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-89b9350e-4c18-4af9-85bd-046508ab5b20 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:38.711694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-89b9350e-4c18-4af9-85bd-046508ab5b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:38.794066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 331/656] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:38.795296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9350e-4c18-4af9-85bd-046508ab5b20 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.806545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbb4c6-0a0a-487b-afa9-9d4e52e4caa5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.808902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.822585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:38.843889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.844753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.846436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.846741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.864850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.866752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:38.887461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:38.888568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:38.889618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:38.889618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:38.889618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.889618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:38.889618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 326/657] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:38.889618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:38.917235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9350e-4c18-4af9-85bd-046508ab5b20 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:38.958641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:38.959694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:38.959973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:38.960245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:38.970350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:38.971827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:38.990319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:38.991367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:38.992272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:38.992272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:38.992272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:38.992272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:38.992272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 332/658] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:38.992560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.007155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eed0a-e209-4cf0-b3ae-95a92d8e2d8b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.043649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.044443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.044727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.044996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.051706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.053159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.062403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:39.066806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:39.067831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 327/659] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:39.068908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.089004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.100809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae5225-18db-4ed2-a69c-96eb540be5fd None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.135028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.135693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.135974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.136772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.138071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:39.138357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.139744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.150914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 333/660] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:39.151738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.175454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3185a-9277-4e08-a881-45b6e6be2005 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.222987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.223797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.224080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.224354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.235024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:39.242822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:39.243778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:39.244614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 328/661] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:39.245506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.270610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086188f2-62ab-43c0-9db2-46711dc40805 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.298422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.305896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.306599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.306889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.307170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.307895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:39.308846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.310525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.323001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 334/662] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:39.325220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.336311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.361524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b30900-524f-4631-934c-8e5745c48cb9 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.397026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.397612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.397896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.398200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.398961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:39.399273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.400679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.412799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 329/663] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:39.413631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.432395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0483f36-7083-4598-a45e-c9f715335680 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.478130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.479206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.479673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.480203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.481594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:39.482106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.484571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.504104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 335/664] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:39.504980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.537664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ce6d0-aa47-4e64-89d5-a3ba28ac073a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.560137 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.576738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.577573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.577855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.578134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.580407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.586730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.588201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:39.604239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:39.606321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:39.607311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 330/665] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:39.609128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.617412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685834a-1d54-4431-84b4-a4cd69261aa6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.662120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.662774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.663004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.663283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.664034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:39.664344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.665737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.673761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 336/666] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:39.674583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.720158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56755-2fcb-41bb-8ede-640617ce7a23 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.753548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.754390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.754675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.754950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.761972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.763605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.774267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0fa35-5d78-4a60-8d95-3b236158801a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:39.777934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:39.779245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 331/667] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:39.780170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.785990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.810602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.811631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.812098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.812471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.819703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.821161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:39.837206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:39.838316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 337/668] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:39.839178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.841112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:39.890546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37560cc-cd7f-496c-8308-97811181261a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.923964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.924541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.924818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.925092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.925919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=28faa0c6ac9b4e3a9528c3199506257c) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:39.926257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.927685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:39.946214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 332/669] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/role_assignments?user.id=28faa0c6ac9b4e3a9528c3199506257c&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:39.946214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:39.946523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2008a-895f-494e-b055-bf26dbee0f9f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:39.979552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:39.980853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:39.981126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:39.981451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:39.982291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:39.982600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:39.984105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.005050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 338/670] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:40.007476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.041230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.058193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336100b-1bf2-439a-b83c-001b615e1af1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.078103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.097204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.098008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.098332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.098603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.115223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.116682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.125236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea7960-5e4d-4b3c-ba3b-1f519df98726 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:40.134420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:40.139568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:40.140547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:40.140547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:40.140547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:40.140547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:40.140547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 333/671] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:40.140547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.166107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.166874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.167158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.167564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.168363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:40.168852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.170334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.182680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 339/672] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:40.184025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.254047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81637c7-383c-4e0e-b6b1-739c2910dda5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.296211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.296944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.297077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.297228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.297502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.298329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:40.298643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.300102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.313514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 334/673] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:40.315087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.320848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2ce1f-dff1-40d2-a9b9-b4f59b6533a4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.348847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.360154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.360818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.361093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.361364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.427775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293881e-019f-417e-93e9-173464cb4df8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.433060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.434712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.465299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.465971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.466305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.466564 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.467287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:40.467832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.468958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.487079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 335/674] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:40.488021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.537092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.554521 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 340/675] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:40.556300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.598026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.608960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c353a98-9a71-44c8-9dcf-0092d2dbabfa None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.648752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.649419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.649707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.649987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.650924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=ac6a657099c14f598d5becd5ab72ee84) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:40.651293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.653295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.670201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 336/676] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/role_assignments?user.id=ac6a657099c14f598d5becd5ab72ee84&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:40.670201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.682731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9f861-2f3a-493e-ad1e-d0447819a6aa None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.725143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.725941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.726327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.726680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.727738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=8d29198c768d4024af04273a3cca8f20) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:40.728131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.729952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.737850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 341/677] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/role_assignments?user.id=8d29198c768d4024af04273a3cca8f20&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:40.738892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.806790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.819378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908e132-5920-4d4b-afc7-e7162c126afb None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.904781 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e6c46-36ed-49b7-b8ed-eecc25d470c5 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:40.905765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.906618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.906900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.907172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.919991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.921489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:40.948222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:40.949986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:40.950943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:40.951484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:40.951484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:40.951484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:40.951484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:40.951759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 337/678] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:40.957970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:40.965484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:40.968096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:40.988862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:40.992732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:40.993596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:40.994515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:40.994515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:40.994515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:40.994515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:40.997027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 342/679] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:41.002518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:41.107617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392854-f5ad-4f81-aac2-4669a7b67b1b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.141726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43261e7-c4cc-4f33-b8cc-78aec6cd2668 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.145453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:41.158410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:41.159099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.162544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.162544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.162544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:41.162544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:41.164628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:41.191293 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:41.191967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.192248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.192546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.192660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 338/680] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Jun 15 14:35:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:41.193249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:41.193585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:41.200347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:41.201469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:41.215685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 343/681] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:41.221342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:41.296860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:41.330011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc70da2-a1f9-412e-a090-2bbf631cec9b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.367662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:41.368422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.368707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.368981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.444299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:41.445883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:41.483633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ae40d-ded0-4f2f-b8ad-c8b71ff54c58 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.523061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:41.558467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 339/682] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:41.563744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:41.570969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:41.571780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.572069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.572343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.594701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:41.596192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:41.626073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:41.627081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:41.627959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:41.634648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 344/683] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:41.643246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:41.673607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdf5de-6dfe-4fff-b839-473f8268fbcd None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.695676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:41.711578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:41.712429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.714382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.714382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.724139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:41.725762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:41.747076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:41.748210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:41.749288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:41.749288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:41.749288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:41.749288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:41.750914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 340/684] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:41.756684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:41.791249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:41.872227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1fe0b-d683-43ca-81c3-b20d8988b737 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.878451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c50b4-338d-494a-b7b6-2cb2f5855156 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:41.909107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:41.909923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:41.910341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:41.910740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:41.911577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:41.911981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:41.913578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:41.927128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 341/685] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:41.932681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:42.014910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:42.015885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.016318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.016727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.017785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:42.018274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:42.019923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:42.034603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:42.046218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42cad5-f9e2-498d-8215-be97fa20c07d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:42.061829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 345/686] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:42.075813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:42.092242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:42.093046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.093326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.093602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.102832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:42.104360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:42.121722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:42.122889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:42.123814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:42.126967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 342/687] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:42.132343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:42.243611 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08274ef1-34c9-464e-bff1-4154f3bdb4dc None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:42.282288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:42.283106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.283388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.283664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.288169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:42.288169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:42.288169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:42.301573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 343/688] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:42.307490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:42.311895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:42.320372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:42.362635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0b081-775d-4e95-b08c-52bc435057e6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:42.420989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84753f74-2e4b-4bf5-8ec7-db46ddda3bdc None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:42.458128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fdb02-ea48-431d-a0b5-28f07e3cca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:42.458366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3f992-7cc9-4454-97cb-d04e337a722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:42.458845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f992-7cc9-4454-97cb-d04e337a722d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.459208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fdb02-ea48-431d-a0b5-28f07e3cca46 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.459473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fdb02-ea48-431d-a0b5-28f07e3cca46 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.459583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f992-7cc9-4454-97cb-d04e337a722d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.459657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fdb02-ea48-431d-a0b5-28f07e3cca46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.459727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f992-7cc9-4454-97cb-d04e337a722d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.460494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fdb02-ea48-431d-a0b5-28f07e3cca46 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=0b3a2fb853764be199cc6ff0dfd18064) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:42.460609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3f992-7cc9-4454-97cb-d04e337a722d None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=bead604b09be482c906a97b53bb03d69) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:42.460810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3f992-7cc9-4454-97cb-d04e337a722d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:42.460907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fdb02-ea48-431d-a0b5-28f07e3cca46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:42.462336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fdb02-ea48-431d-a0b5-28f07e3cca46 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:42.470612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3f992-7cc9-4454-97cb-d04e337a722d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:42.470904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 344/689] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:42 2022] GET /identity/v3/role_assignments?user.id=0b3a2fb853764be199cc6ff0dfd18064&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:42.486515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 346/690] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:42 2022] GET /identity/v3/role_assignments?user.id=bead604b09be482c906a97b53bb03d69&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 608 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:42.534197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bffe4-54c6-41c5-9397-b6e8af7a1743 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.534197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bffe4-54c6-41c5-9397-b6e8af7a1743 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.534197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bffe4-54c6-41c5-9397-b6e8af7a1743 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.534197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 345/691] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:42.539923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922bfe-b22d-468a-af8c-c858ed69660f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.540409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922bfe-b22d-468a-af8c-c858ed69660f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.540868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922bfe-b22d-468a-af8c-c858ed69660f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.571548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6f922bfe-b22d-468a-af8c-c858ed69660f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:42.572899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6f922bfe-b22d-468a-af8c-c858ed69660f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:42.670258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 347/692] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:42.704525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bffe4-54c6-41c5-9397-b6e8af7a1743 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:42.784234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:42.830310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bffe4-54c6-41c5-9397-b6e8af7a1743 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:42.869179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:42.869980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:42.870381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:42.870817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:42.882099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:42.883605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:42.901989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:42.903117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:42.904013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:42.904013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:42.904013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:42.904013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:42.904013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 346/693] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:42.909752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f922bfe-b22d-468a-af8c-c858ed69660f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:43.006806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:43.038328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f922bfe-b22d-468a-af8c-c858ed69660f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:43.088239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df8f0e-6d32-4e5c-836b-0ac542764967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:43.088937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df8f0e-6d32-4e5c-836b-0ac542764967 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:43.089855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df8f0e-6d32-4e5c-836b-0ac542764967 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:43.090266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df8f0e-6d32-4e5c-836b-0ac542764967 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:43.091159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df8f0e-6d32-4e5c-836b-0ac542764967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:43.091474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df8f0e-6d32-4e5c-836b-0ac542764967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:43.093043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df8f0e-6d32-4e5c-836b-0ac542764967 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:43.113885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 348/694] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:43.119450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:43.272312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef82b7-761f-4dc0-92bb-e12cf5ce5a73 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:43.294136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:43.314606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e892a-a2c2-4c6f-9633-32d8c79e0f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:43.315288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e892a-a2c2-4c6f-9633-32d8c79e0f2e None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:43.315567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e892a-a2c2-4c6f-9633-32d8c79e0f2e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:43.315839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e892a-a2c2-4c6f-9633-32d8c79e0f2e None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:43.316549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e892a-a2c2-4c6f-9633-32d8c79e0f2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:43.317998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e892a-a2c2-4c6f-9633-32d8c79e0f2e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:43.412500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 347/695] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:35:43 2022] POST /identity/v3/users => generated 254 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:43.506873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13ac3-eaca-42ee-a6eb-803a0475a08a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:43.507208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13ac3-eaca-42ee-a6eb-803a0475a08a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:43.507426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13ac3-eaca-42ee-a6eb-803a0475a08a None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:43.509152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 349/696] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:43.515337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785dd6-b461-4045-87ee-28bdb9fe7e45 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:43.515664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785dd6-b461-4045-87ee-28bdb9fe7e45 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:43.515892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785dd6-b461-4045-87ee-28bdb9fe7e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:43.544344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-79785dd6-b461-4045-87ee-28bdb9fe7e45 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:43.545553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-79785dd6-b461-4045-87ee-28bdb9fe7e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:43.653344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 348/697] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:43.690826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b13ac3-eaca-42ee-a6eb-803a0475a08a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:43.814904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b13ac3-eaca-42ee-a6eb-803a0475a08a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:43.836779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db546c78-cffa-4235-867c-6a96dcac58fe None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:43.837116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db546c78-cffa-4235-867c-6a96dcac58fe None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:43.837701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db546c78-cffa-4235-867c-6a96dcac58fe None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:43.844105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 349/698] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:43.846767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b694d-f239-4611-90d5-1e2e6adcec0d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:43.847057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b694d-f239-4611-90d5-1e2e6adcec0d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:43.847341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b694d-f239-4611-90d5-1e2e6adcec0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:43.863041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:43.864261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:43.864720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:43.865696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:43.872668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6c8b694d-f239-4611-90d5-1e2e6adcec0d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:43.874841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6c8b694d-f239-4611-90d5-1e2e6adcec0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:43.875215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:43.882297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:43.891845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:43.892863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:43.893867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:43.896334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 350/699] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:43.902324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:43.968592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 350/700] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:44.002946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b694d-f239-4611-90d5-1e2e6adcec0d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:44.040921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e292936-9f54-41df-90e6-d50a053e5da2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:44.066119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:44.087248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:44.087948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.088236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.088517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.089727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:44.090053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:44.091527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:44.117484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 351/701] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:44.128784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:44.170581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b694d-f239-4611-90d5-1e2e6adcec0d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:44.207796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:44.208618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.209165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.209444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.210111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:44.211551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:44.232498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:44.237251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:44.238079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 351/702] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Jun 15 14:35:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:44.243359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:44.252360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:44.386515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea54641-e27f-491a-bc2c-4e2137aba976 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:44.390520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:44.408197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941c3c3-33d9-443b-9389-0f1a5acaae49 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:44.426563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:44.427235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.427516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.427794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.428504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:44.428811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:44.430226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:44.438334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 352/703] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Jun 15 14:35:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:44.442964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:44.479916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:44.481266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.482656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.483314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.505963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:44.509063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:44.547744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:44.548804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:44.549687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:44.549687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:44.549687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:44.549687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:44.549687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 352/704] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:44.562229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:44.722207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d73894-de51-42db-8b0d-0217c945f508 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:44.728963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:44.762042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:44.762796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.763083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.763414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.764161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:44.764667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:44.766405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:44.793569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 353/705] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:44.798959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:44.826229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2498493-86ca-4a6e-89bf-2df3b05fe94f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:44.901565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:44.902306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:44.902714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:44.902994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:44.903720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:44.904047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:44.905547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:44.926764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 353/706] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Jun 15 14:35:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:44.965087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c98aa-11c6-46d0-8bf9-2e5a573d93d1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:45.041863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496d59b-04ff-4265-9fe0-d556d0431225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:45.043600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d59b-04ff-4265-9fe0-d556d0431225 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.043811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d59b-04ff-4265-9fe0-d556d0431225 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.044212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d59b-04ff-4265-9fe0-d556d0431225 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.053948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496d59b-04ff-4265-9fe0-d556d0431225 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:45.055792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496d59b-04ff-4265-9fe0-d556d0431225 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-4496d59b-04ff-4265-9fe0-d556d0431225 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:45.072354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:45.073732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:45.074717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:45.077504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 354/707] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:35:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:45.087365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:45.134056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:45.235493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44db9f-de9a-4f1f-9ffe-56a0ebad3bc7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:45.239459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683860-4879-4886-8662-bc6c98f1e6d4 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.240007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683860-4879-4886-8662-bc6c98f1e6d4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.240528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683860-4879-4886-8662-bc6c98f1e6d4 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.242422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 355/708] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:45.257269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490b55e-6e04-407a-b608-f2a37cc97525 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.257902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490b55e-6e04-407a-b608-f2a37cc97525 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.258407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490b55e-6e04-407a-b608-f2a37cc97525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.275707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:45.277559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.277850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.278192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.279133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:45.279612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:45.281067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:45.307612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-5490b55e-6e04-407a-b608-f2a37cc97525 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:45.309645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5490b55e-6e04-407a-b608-f2a37cc97525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:45.317838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 354/709] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:35:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:45.323874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:45.364354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:45.434587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 356/710] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:45.436037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133f090-6421-451e-9788-82c512e88568 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.436330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133f090-6421-451e-9788-82c512e88568 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.436609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133f090-6421-451e-9788-82c512e88568 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.438285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 357/711] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:45.460099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1d14-f8a2-40c3-8dac-72ebf94d83b5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.460374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1d14-f8a2-40c3-8dac-72ebf94d83b5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.460651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1d14-f8a2-40c3-8dac-72ebf94d83b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.481543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:45.482697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-4cdd1d14-f8a2-40c3-8dac-72ebf94d83b5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:45.484119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4cdd1d14-f8a2-40c3-8dac-72ebf94d83b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:45.498663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a315-8632-40d2-910b-5ec6259e8314 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:45.536799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:45.537808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.538442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.538723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.590666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 358/712] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 14:35:45.591926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1d14-f8a2-40c3-8dac-72ebf94d83b5 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:45.650303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:45.652637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:45.696041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:45.705930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1d14-f8a2-40c3-8dac-72ebf94d83b5 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:45.744238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:45.745254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.745982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.746270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] PATH_INFO: `/v3/services/network` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.747059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:45.748552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:45.749950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:45.767254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:45.774613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:45.775467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:45.776291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 355/714] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 14:35:45.776291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:45.777696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 359/714] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:45 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:45.778961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:45.918130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2069c5-0ba9-43ec-b8c2-9f9214ad5a7e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:45.925179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31225e-607f-4136-a718-0b6457e04d65 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:45.970804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:45.971962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.972451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.972960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.979207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:45.981152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:45.981942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:45.982433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:45.984181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:45.984700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:45.986766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:45.987471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:45.995907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:45.997008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:45.997850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:46.003428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 360/715] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:35:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:46.003428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.003561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 356/716] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:46.004294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.033434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:46.063885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:46.136552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b88cf-42e5-431a-8d19-28635e4c74f4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:46.142542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b88974-9b00-4796-a766-2cd6cda6be1b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:46.183334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:46.184428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.185131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.185621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.186580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] RBAC: Adding query filter params (type=network) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:46.187118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:46.188246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:46.188991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:46.189100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.189324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.189684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.202620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 357/717] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:35:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:46.203888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:46.204172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.205640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:46.228339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:46.234832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:46.235726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:46.235726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:46.235726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:46.235726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:46.235726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 361/718] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:46.235726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.360068 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b02ce0-12c5-42f0-b79a-19f836856b8d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:46.380093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:46.428037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:46.428749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.429032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.429312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.430107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:46.430448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:46.433572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:46.445627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 358/719] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:46.452334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.455234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:46.490867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795839d9-6141-49ce-b063-4192e4b20f63 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:46.536169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2990f-2173-4843-98e0-72ed284ae125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:46.537256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2990f-2173-4843-98e0-72ed284ae125 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.537693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2990f-2173-4843-98e0-72ed284ae125 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.538109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2990f-2173-4843-98e0-72ed284ae125 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.538890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2990f-2173-4843-98e0-72ed284ae125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:46.539207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2990f-2173-4843-98e0-72ed284ae125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:46.540905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2990f-2173-4843-98e0-72ed284ae125 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:46.564720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 362/720] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:46.566094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cc91-b37e-455e-9720-b19dbc9b0708 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.566411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cc91-b37e-455e-9720-b19dbc9b0708 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.566695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cc91-b37e-455e-9720-b19dbc9b0708 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.568181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 363/721] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:46.572904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cc91-b37e-455e-9720-b19dbc9b0708 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.598678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643f9b7-c46d-4b3b-a8da-ada91e2e6f15 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:46.659868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f50cb3-4276-4cb1-a909-1a349927652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:46.660706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f50cb3-4276-4cb1-a909-1a349927652d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.660984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f50cb3-4276-4cb1-a909-1a349927652d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.661256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f50cb3-4276-4cb1-a909-1a349927652d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.679284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f50cb3-4276-4cb1-a909-1a349927652d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:46.680769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f50cb3-4276-4cb1-a909-1a349927652d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:46.702602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cc91-b37e-455e-9720-b19dbc9b0708 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-b3f50cb3-4276-4cb1-a909-1a349927652d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:46.707742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:46.709199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:46.710285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:46.710285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:46.710285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:46.710285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:46.711550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 359/722] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:46.712988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951ab21-4e2e-4c5a-8d85-a32f2ec8ae21 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.713347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951ab21-4e2e-4c5a-8d85-a32f2ec8ae21 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.713699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951ab21-4e2e-4c5a-8d85-a32f2ec8ae21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.738083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5951ab21-4e2e-4c5a-8d85-a32f2ec8ae21 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:46.739725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5951ab21-4e2e-4c5a-8d85-a32f2ec8ae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:46.744084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:46.751756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:46.752847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.753314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.753798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.761994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:46.763921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:46.769303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:46.780812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:46.781757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:46.782679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:46.785118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 364/723] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:46.786249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.850439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 360/724] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:46.852027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951ab21-4e2e-4c5a-8d85-a32f2ec8ae21 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:46.930012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e1f25-0cea-4531-bf84-6323976b4ee6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:46.977898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:46.979286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:46.979778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:46.980256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:46.981943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:46.982494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:46.985013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:46.989725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951ab21-4e2e-4c5a-8d85-a32f2ec8ae21 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.001933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 365/725] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:47.003570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.029996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.030751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.031033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.031307 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.032091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:47.032402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.033959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.048582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 361/726] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:47.050234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.067666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:47.074932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:47.144153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0233c79-3ce3-4a03-9be9-aab73fcf882a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.185178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb25ee-a4b2-4bcb-bbff-070c0a8171e8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.194060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.195841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.196333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.197309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.212228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:47.225079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:47.226075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:47.226960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:47.235327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.235327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.235327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.235327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.235720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 366/727] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:47.236743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.242447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.244101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:47.264716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:47.270820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 362/728] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:47.271751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.370114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:47.389596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ce42-992f-45dd-b268-1e25395c03b3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.403386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:47.411850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063b7db-771e-46ac-80e4-db6f1e378e95 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.429388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.430065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.430688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.430966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.431805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=ac6a657099c14f598d5becd5ab72ee84) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:47.432126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.434482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.450205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 367/729] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/role_assignments?user.id=ac6a657099c14f598d5becd5ab72ee84&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:47.450205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.462274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d3f88-1d22-4a4d-bdc7-4fedd5aad5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.463138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d3f88-1d22-4a4d-bdc7-4fedd5aad5f4 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.463421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d3f88-1d22-4a4d-bdc7-4fedd5aad5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.463733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d3f88-1d22-4a4d-bdc7-4fedd5aad5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.464505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d3f88-1d22-4a4d-bdc7-4fedd5aad5f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:47.464819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d3f88-1d22-4a4d-bdc7-4fedd5aad5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.466467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d3f88-1d22-4a4d-bdc7-4fedd5aad5f4 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.482562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 363/730] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:47.486960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a2f9c-35cd-4470-a1d8-570e3f532131 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.487348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a2f9c-35cd-4470-a1d8-570e3f532131 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.487806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a2f9c-35cd-4470-a1d8-570e3f532131 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.493924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 364/731] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:47.495550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a2f9c-35cd-4470-a1d8-570e3f532131 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.589525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d881dc7-4c5a-4e39-9b33-431886fd159a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.633279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90ff3d-a385-4007-adca-c1565484f6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.634004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90ff3d-a385-4007-adca-c1565484f6e9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.634331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90ff3d-a385-4007-adca-c1565484f6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.634676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90ff3d-a385-4007-adca-c1565484f6e9 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.635549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90ff3d-a385-4007-adca-c1565484f6e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:47.636004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90ff3d-a385-4007-adca-c1565484f6e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.637488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90ff3d-a385-4007-adca-c1565484f6e9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.662234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 368/732] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:47.662234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cec48-d286-4c99-a5a9-012704d515a5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.662234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cec48-d286-4c99-a5a9-012704d515a5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.662234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cec48-d286-4c99-a5a9-012704d515a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.677025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a2f9c-35cd-4470-a1d8-570e3f532131 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.678942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7b7cec48-d286-4c99-a5a9-012704d515a5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:47.680346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7b7cec48-d286-4c99-a5a9-012704d515a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:47.713258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:47.721268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.723292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.723782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.737293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.737293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:47.739678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.741154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.769075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 369/733] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:47.769368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:47.770291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:47.771172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:47.771172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:47.771172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:47.771172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:47.772786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cec48-d286-4c99-a5a9-012704d515a5 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.779163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 365/734] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:47.785330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:47.914066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125144e-fe3e-498e-8412-494dd316ab5b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.924523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cec48-d286-4c99-a5a9-012704d515a5 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:47.957567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.958284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.958573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.959120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.959855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:47.960232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:47.961603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:47.972637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:47.973699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:47.974186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:47.978855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:47.978983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 366/735] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:47.978983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:48.017413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:48.044539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:48.061955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:48.063689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:48.130086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e521d-3aac-4dad-b5f2-012bce5b0417 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:48.169489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:48.170347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:48.170638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:48.170914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:48.171651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:48.173131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:48.191311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 370/736] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:48.192160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:48.270599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 367/737] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:35:47 2022] POST /identity/v3/services => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:48.272064 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:48.309038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:48.456200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0342a2-f56c-4ef9-809e-4ca4f4c5867d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:48.488403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b562794-0056-4aef-a007-572e33b1626f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:48.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:48.541252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:48.541252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:48.541252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:48.541252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:48.554486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:48.556795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:48.582231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:48.583203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:48.584366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:48.584366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:48.584366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:48.584366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:48.586312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 368/738] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:48.588175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:48.589994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:48.590870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:48.591155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:48.591606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:48.605011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:48.607439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:48.625328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:48.627203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:48.629045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 371/739] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:35:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:48.634951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:48.762767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9137b-e60f-4237-b978-eb4c5d327fd1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:48.779833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7b08b-126e-4f48-948c-a14cd5ff1e9e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:48.792734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:48.803901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce8a2d-e53e-4e79-bc2f-e9f84b4c8e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:48.804763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8a2d-e53e-4e79-bc2f-e9f84b4c8e6e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:48.805112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8a2d-e53e-4e79-bc2f-e9f84b4c8e6e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:48.805448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8a2d-e53e-4e79-bc2f-e9f84b4c8e6e None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:48.806583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8a2d-e53e-4e79-bc2f-e9f84b4c8e6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:48.806963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8a2d-e53e-4e79-bc2f-e9f84b4c8e6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:48.809163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8a2d-e53e-4e79-bc2f-e9f84b4c8e6e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:48.826255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09501737-f5cf-4c07-87a2-e4d37882eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:48.826940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501737-f5cf-4c07-87a2-e4d37882eb30 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:48.827810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501737-f5cf-4c07-87a2-e4d37882eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:48.828102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501737-f5cf-4c07-87a2-e4d37882eb30 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:48.828967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09501737-f5cf-4c07-87a2-e4d37882eb30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:48.829282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09501737-f5cf-4c07-87a2-e4d37882eb30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:48.830162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 372/740] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Jun 15 14:35:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:48.835844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09501737-f5cf-4c07-87a2-e4d37882eb30 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:48.836073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bbded-23c0-4a59-983b-77be5fe2c156 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:48.836360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bbded-23c0-4a59-983b-77be5fe2c156 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:48.836728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bbded-23c0-4a59-983b-77be5fe2c156 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:48.838860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 373/741] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:48.843780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bbded-23c0-4a59-983b-77be5fe2c156 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:48.854586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 369/742] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:48.856183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518520-b73f-41c0-9202-4499677a821a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:48.856477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518520-b73f-41c0-9202-4499677a821a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:48.856759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518520-b73f-41c0-9202-4499677a821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:48.880271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8e518520-b73f-41c0-9202-4499677a821a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:48.881500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8e518520-b73f-41c0-9202-4499677a821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:48.896087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:48.977073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bbded-23c0-4a59-983b-77be5fe2c156 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:48.979965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 370/743] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 14:35:48.982215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e518520-b73f-41c0-9202-4499677a821a None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:49.017388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.018012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.018345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.018621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.081465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:49.105018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.106618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.122757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e518520-b73f-41c0-9202-4499677a821a None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:49.161274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.162270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.162589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.162875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.170958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.172458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:49.188352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:49.189296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:49.190207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:49.194305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 371/744] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:49.195219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:49.206563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:49.238307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 374/745] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:48 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 14:35:49.239450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:49.366903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ccf7-bfaf-4b42-835a-4e16e6713432 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:49.393927 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:49.416459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.418022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d801b4-4fd9-43d8-9302-109b5eb589e0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:49.418358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.418811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.419246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.420329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.424044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.460855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.461696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.462078 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.462464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.463699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:49.464344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.466439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.484357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 375/746] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:49.484357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:49.539927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 372/747] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:35:49 2022] POST /identity/v3/services => generated 240 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:49.541269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:49.626510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:49.630786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c73ea-6bf0-4bed-805b-2f4e05b71840 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:49.682699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c71719-2718-4b3b-a0ca-d73f56fd171b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:49.688590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.689465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.689747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.690017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.715261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.716761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.721819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:49.749254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:49.750246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:49.751202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:49.751202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:49.751202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:49.751202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:49.754914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 376/748] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:49.756963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:49.774148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af5428-a885-46fb-9061-58e6de762753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.774892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af5428-a885-46fb-9061-58e6de762753 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.775299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af5428-a885-46fb-9061-58e6de762753 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.775593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af5428-a885-46fb-9061-58e6de762753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.777050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af5428-a885-46fb-9061-58e6de762753 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=0b3a2fb853764be199cc6ff0dfd18064) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:49.777371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af5428-a885-46fb-9061-58e6de762753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.781806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af5428-a885-46fb-9061-58e6de762753 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.796629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 373/749] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:49 2022] GET /identity/v3/role_assignments?user.id=0b3a2fb853764be199cc6ff0dfd18064&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:49.891774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ddcf0-ca7d-47db-9ab2-50c40f087801 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.891774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ddcf0-ca7d-47db-9ab2-50c40f087801 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.892020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ddcf0-ca7d-47db-9ab2-50c40f087801 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.895281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 374/750] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:49.899190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396de0-e2ec-42a3-a118-e4e842de4aa7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:49.901284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc1d8-959c-4247-8ca7-fe20f333efc9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.901562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc1d8-959c-4247-8ca7-fe20f333efc9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.901905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc1d8-959c-4247-8ca7-fe20f333efc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.929077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1c0cc1d8-959c-4247-8ca7-fe20f333efc9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:49.931078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1c0cc1d8-959c-4247-8ca7-fe20f333efc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:49.946130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:49.947443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:49.947443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:49.947704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:49.949307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:49.949674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:49.951443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:49.969804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 377/751] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:49.976404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:50.050366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 375/752] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:35:50.059912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:50.081128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc1d8-959c-4247-8ca7-fe20f333efc9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:50.136606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f963e54-5b9f-41a7-a161-c16ae0139958 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:50.166733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:50.217687 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc1d8-959c-4247-8ca7-fe20f333efc9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:50.260663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:50.261684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:50.262033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:50.262344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:50.268191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:50.268996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:50.269273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:50.269544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:50.280061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:50.303885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:50.304017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:50.304804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:50.305703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:50.310740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:50.311635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:50.312518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:50.312934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 376/753] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:50.313451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:50.314289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 378/754] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:50.323770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:50.567574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aa14d-bb3c-4460-b2af-5e834e83817c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:50.574132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a4542-d70f-4e03-a303-99291ed39945 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:50.598825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:50.610724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:50.640184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:50.640866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:50.641152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:50.641430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:50.642260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:50.642583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:50.644038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:50.655557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:50.656247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:50.656529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:50.656804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:50.657563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:50.657869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:50.659277 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:50.662635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 379/755] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:50.663935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:50.684873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 377/756] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:50.703279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:50.785631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ee77e-df15-4f25-a053-2b1c2d81628d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:50.843473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:50.844160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:50.844443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:50.844718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:50.845574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=fd1b940712c1481ba53ae1b45c68c347) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:50.845889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:50.851234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:50.870344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 380/757] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:50 2022] GET /identity/v3/role_assignments?user.id=fd1b940712c1481ba53ae1b45c68c347&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:50.938449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c39b9e-93c7-493f-b536-e6880ea821ef None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:50.993612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059590c-56ce-4301-a75e-269d9a8e1749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:50.994471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059590c-56ce-4301-a75e-269d9a8e1749 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:50.995341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059590c-56ce-4301-a75e-269d9a8e1749 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:50.995774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059590c-56ce-4301-a75e-269d9a8e1749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.010946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059590c-56ce-4301-a75e-269d9a8e1749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:51.012468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059590c-56ce-4301-a75e-269d9a8e1749 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-6059590c-56ce-4301-a75e-269d9a8e1749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:51.039583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:51.040520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:51.041550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:51.041550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:51.041550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:51.041550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:51.046236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 378/758] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:51.047728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.059071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9ee36-d377-443f-b671-553a1f5ea3e2 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.059410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9ee36-d377-443f-b671-553a1f5ea3e2 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.059626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9ee36-d377-443f-b671-553a1f5ea3e2 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.061526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 379/759] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:51.080205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f4c08-7fb8-4713-b8cd-0eaa5bd2131b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.080522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f4c08-7fb8-4713-b8cd-0eaa5bd2131b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.080772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f4c08-7fb8-4713-b8cd-0eaa5bd2131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.103420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:51.119402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-190f4c08-7fb8-4713-b8cd-0eaa5bd2131b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:51.121317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-190f4c08-7fb8-4713-b8cd-0eaa5bd2131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:51.172141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:51.180539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af271b-8da2-4422-b464-b99a4675cc3f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:51.246882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 380/760] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:51.255894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:51.256595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.256885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.257163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.257909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:51.258239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:51.259639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:51.274728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 381/761] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:51.280249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f4c08-7fb8-4713-b8cd-0eaa5bd2131b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.309040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.442983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:51.449797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f8e3a-7047-4bb4-a36c-eea0d81d1a8c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:51.485780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f4c08-7fb8-4713-b8cd-0eaa5bd2131b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:51.522813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:51.523619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.523902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.524182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.534197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:51.534197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:51.547856 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:51.551170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:51.551312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:51.552217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:51.553160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:51.553696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.553696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.553696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:51.553870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 381/762] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:51.558514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:51.562224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:51.575701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:51.576739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:51.577577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:51.580495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 382/763] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:51.587227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.619013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:51.719367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6828a4-b0ad-4eb3-88ca-1a762be4a757 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:51.770762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6850cf-85f7-4855-8592-3ccc00dbd30b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:51.785870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:51.786685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.786976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.787259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.787988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:51.788303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:51.789936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:51.804298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 383/764] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:51.810591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.812901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:51.813539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:51.813825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:51.814255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:51.815125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:51.815435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:51.816880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:51.836357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 382/765] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:51.842770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:51.874612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.009610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.032319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2101dbe-a187-425f-97d4-ea9a708c09e3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.052501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d62be-b35b-4a88-a291-92c1a21c8eaa None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.090395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.091192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.091480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.091806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.102730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.104748 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:52.124027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:52.125003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:52.125915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:52.125915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:52.125915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:52.125915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:52.125915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 383/766] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:52.127007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.127698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.127979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.128251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.128972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:52.129278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.130720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.131239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:52.148817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 384/767] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Jun 15 14:35:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:52.168133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:52.257270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49935ea2-9dc7-4875-86f5-c3b3e4cb1de1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.292652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.298629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b012-a809-462b-8a6e-941d8f78080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.299554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b012-a809-462b-8a6e-941d8f78080c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.300090 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b012-a809-462b-8a6e-941d8f78080c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.300569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b012-a809-462b-8a6e-941d8f78080c None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.301503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b012-a809-462b-8a6e-941d8f78080c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:52.301999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b012-a809-462b-8a6e-941d8f78080c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.303586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b012-a809-462b-8a6e-941d8f78080c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.316567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 384/768] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:52.318282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71ef96-cd16-4350-a07f-69f92c2a3239 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.318732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.319455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71ef96-cd16-4350-a07f-69f92c2a3239 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.319924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71ef96-cd16-4350-a07f-69f92c2a3239 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.321721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 385/769] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:52.323148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71ef96-cd16-4350-a07f-69f92c2a3239 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:52.335967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5913-7cfd-4fac-9c43-11d2b466e419 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.387985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7012e5-f21b-437b-bfe0-76a879ff74c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.388678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7012e5-f21b-437b-bfe0-76a879ff74c1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.389022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7012e5-f21b-437b-bfe0-76a879ff74c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.389363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7012e5-f21b-437b-bfe0-76a879ff74c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.390527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7012e5-f21b-437b-bfe0-76a879ff74c1 None admin] RBAC: Adding query filter params (interface=public, service_id=36eeb523aa314eaeb107476aeed65ad9, region_id=RegionOne) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:52.391310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7012e5-f21b-437b-bfe0-76a879ff74c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.393046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7012e5-f21b-437b-bfe0-76a879ff74c1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.408117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 385/770] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:35:52 2022] GET /identity/v3/endpoints?service_id=36eeb523aa314eaeb107476aeed65ad9&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:52.408117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4593e-f620-43db-9e09-def8c4ac05f9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.408117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4593e-f620-43db-9e09-def8c4ac05f9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.408117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4593e-f620-43db-9e09-def8c4ac05f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.427119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-deb4593e-f620-43db-9e09-def8c4ac05f9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:52.429183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-deb4593e-f620-43db-9e09-def8c4ac05f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:52.453526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71ef96-cd16-4350-a07f-69f92c2a3239 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.504817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.505783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.506112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.506480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.520030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.522392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.530253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 386/771] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:52.532065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4593e-f620-43db-9e09-def8c4ac05f9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:52.550544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:52.551482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:52.552396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:52.556901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 386/772] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:52.558242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:52.634475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.657817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.678026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4593e-f620-43db-9e09-def8c4ac05f9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.699145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b033bbc-6521-4e64-8065-ad06b09ed430 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.719762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.720553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.720834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.721105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.721795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.723221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.730548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 387/773] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:35:52 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:52.731463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:52.751646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9657c-0845-4296-a69d-8c1d82d66b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.753024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9657c-0845-4296-a69d-8c1d82d66b5d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.753541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9657c-0845-4296-a69d-8c1d82d66b5d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.754213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9657c-0845-4296-a69d-8c1d82d66b5d None admin] PATH_INFO: `/v3/services/network` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.755360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9657c-0845-4296-a69d-8c1d82d66b5d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.757065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9657c-0845-4296-a69d-8c1d82d66b5d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-3fc9657c-0845-4296-a69d-8c1d82d66b5d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:52.773606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:52.774739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:52.775705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:52.778684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 387/774] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:52 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:52.780515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab64ae-6a52-472d-ad2f-6eb44270e0cb None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.781023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab64ae-6a52-472d-ad2f-6eb44270e0cb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.781542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab64ae-6a52-472d-ad2f-6eb44270e0cb None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.783453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 388/775] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:52.785423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab64ae-6a52-472d-ad2f-6eb44270e0cb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:52.873544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d53ef8-dac5-4a67-9917-982e9bf8ea87 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.916512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399ebdc-3cb2-4dd7-973d-5bf26c558499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.918047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399ebdc-3cb2-4dd7-973d-5bf26c558499 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.918550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399ebdc-3cb2-4dd7-973d-5bf26c558499 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.918886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399ebdc-3cb2-4dd7-973d-5bf26c558499 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.919791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399ebdc-3cb2-4dd7-973d-5bf26c558499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:52.920203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399ebdc-3cb2-4dd7-973d-5bf26c558499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.922080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399ebdc-3cb2-4dd7-973d-5bf26c558499 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.922999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab64ae-6a52-472d-ad2f-6eb44270e0cb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:52.942022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 388/776] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:52.942877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.944812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15382-add5-48a5-a396-0835abf67e51 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.945157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15382-add5-48a5-a396-0835abf67e51 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.945440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15382-add5-48a5-a396-0835abf67e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.962992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:52.964444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:52.976666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:52.976666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:52.976666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] RBAC: Adding query filter params (name=network) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:52.976666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:52.976666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:52.977035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-60f15382-add5-48a5-a396-0835abf67e51 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:52.977035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-60f15382-add5-48a5-a396-0835abf67e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:52.977035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:52.977237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 389/777] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:35:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:52.977237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.066551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 389/778] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:53.067764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15382-add5-48a5-a396-0835abf67e51 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.113468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f721-afcc-496b-9c17-e811b403218d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:53.150504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:53.152324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.152840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.153343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/ac6a657099c14f598d5becd5ab72ee84/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.155601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=ac6a657099c14f598d5becd5ab72ee84, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:53.157335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:53.202395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15382-add5-48a5-a396-0835abf67e51 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:53.245058 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 390/779] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:52 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/ac6a657099c14f598d5becd5ab72ee84/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:53.246756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.258514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:53.261586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.262279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.265526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.267515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] RBAC: Adding query filter params (type=network) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:53.269709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:53.272718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:53.285790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 390/780] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:35:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:53.294727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.300503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:53.345526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:53.432927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3b823-7845-4e5b-a104-5cfffbe9974e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:53.446644 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9e976-7c92-4b91-804d-2f4405d45b88 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:53.483570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8f0d4-a643-42df-b91e-52f93d751428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:53.484383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f0d4-a643-42df-b91e-52f93d751428 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.484669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f0d4-a643-42df-b91e-52f93d751428 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.484941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f0d4-a643-42df-b91e-52f93d751428 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.490162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb74a4-f7b2-4a59-82cc-2219f4a81edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:53.490885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb74a4-f7b2-4a59-82cc-2219f4a81edc None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.491167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb74a4-f7b2-4a59-82cc-2219f4a81edc None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.491440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb74a4-f7b2-4a59-82cc-2219f4a81edc None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.492236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb74a4-f7b2-4a59-82cc-2219f4a81edc None admin] RBAC: Adding query filter params (interface=public, service_id=f31e5f4861b94f9fa98ee706e2c9c106, region_id=RegionOne) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:53.492555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb74a4-f7b2-4a59-82cc-2219f4a81edc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:53.493936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb74a4-f7b2-4a59-82cc-2219f4a81edc None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:53.497828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8f0d4-a643-42df-b91e-52f93d751428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:53.501285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 391/781] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:35:53 2022] GET /identity/v3/endpoints?service_id=f31e5f4861b94f9fa98ee706e2c9c106&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:53.502655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95c98b-bbdf-4fc9-a623-a48d1097f6f4 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.502944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95c98b-bbdf-4fc9-a623-a48d1097f6f4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.503222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95c98b-bbdf-4fc9-a623-a48d1097f6f4 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.504715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 392/782] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-5fc8f0d4-a643-42df-b91e-52f93d751428 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:53.506754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:53.507820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:53.508801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:53.509560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95c98b-bbdf-4fc9-a623-a48d1097f6f4 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.512794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 391/783] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:53.514257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e53e-ad35-4a5e-a3ea-275a365e364e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.514542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e53e-ad35-4a5e-a3ea-275a365e364e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.514818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e53e-ad35-4a5e-a3ea-275a365e364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.537288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7f36e53e-ad35-4a5e-a3ea-275a365e364e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:53.538403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7f36e53e-ad35-4a5e-a3ea-275a365e364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:53.597631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:53.636779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 392/784] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:53.638453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e53e-ad35-4a5e-a3ea-275a365e364e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.642996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95c98b-bbdf-4fc9-a623-a48d1097f6f4 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:53.673373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:53.698961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:53.700241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.700241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.700514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.701204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:53.702723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:53.718213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 393/785] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:35:53 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:53.718213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.823890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e53e-ad35-4a5e-a3ea-275a365e364e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:53.870568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:53.871354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:53.871638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:53.871918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:53.872686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:53.872995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:53.874444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:53.899735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 393/786] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:53.904271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:53.941509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b50fa1-ec05-4fd1-9939-cdac40681421 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:53.989177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:53.999952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.003286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.003286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.003286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.015633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:54.024383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:54.025357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:35:54.026875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:54.031956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 394/787] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:35:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:54.038289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.060221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:54.078689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0ab71-87f2-45aa-a3b1-7ba98a9f8dd9 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:54.115723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.116525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.116805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.117079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.132943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:54.134608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:54.160076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:54.161417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:54.165501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:54.165501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:54.165501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:54.165501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:54.165501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 394/788] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:54.173131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.199006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5824d-f737-43e3-9daf-c227d195a6b3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:54.281240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.281923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.282231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.282513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.283383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:54.283801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:54.285223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:54.293586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fd40c-2e2c-4314-b631-83d4b01e7102 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:54.309642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 395/789] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:54.314986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.352858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.353459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.354340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.354624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.355513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:54.355887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:54.357331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:54.371158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 395/790] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:54.376846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.382480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:54.457988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:54.515432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46deeff8-ae3a-40ef-9eb6-65c9abe419c7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:54.571219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.572824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.573236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.573520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.586995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:54.589495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:54.611527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:54.612514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:54.613421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:54.613421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:54.613421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:54.613421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:54.613421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 396/791] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:54.619881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.649483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f3ce4-990e-4675-95d9-fd8aaab8fc23 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:54.718711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.720075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.721323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.722440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.737561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:54.740406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:54.766827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:54.767865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:35:54.768832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:54.781088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 396/792] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:54.786788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.804568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadf11-9386-4b00-8549-5c0ff03026ca None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:54.810748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:54.874505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:54.875536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:54.876045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:54.876535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:54.877472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:54.878039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:54.879669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:54.906091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 397/793] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:54.912482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:54.914489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:54.995081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1639414-045b-45d0-b2c5-41d630931b53 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.046207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.051275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.051799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.052284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.053412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:55.053928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.055633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.088108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 397/794] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:55.094458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.101551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888371b5-0659-4a5f-a605-bbfd30de85b3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.153807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68c2c0-05db-4ac0-b41b-11c72401b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.154594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c2c0-05db-4ac0-b41b-11c72401b352 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.154879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c2c0-05db-4ac0-b41b-11c72401b352 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.155146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c2c0-05db-4ac0-b41b-11c72401b352 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.169118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c2c0-05db-4ac0-b41b-11c72401b352 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.171157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c2c0-05db-4ac0-b41b-11c72401b352 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-6d68c2c0-05db-4ac0-b41b-11c72401b352 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:55.201497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:55.202558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:35:55.203487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:55.204022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 398/795] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:35:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:55.211622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10005486-70f8-4ef7-b5e2-5b0ec28698ac None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.211905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10005486-70f8-4ef7-b5e2-5b0ec28698ac None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.212179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10005486-70f8-4ef7-b5e2-5b0ec28698ac None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.214376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 399/796] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:55.214920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10005486-70f8-4ef7-b5e2-5b0ec28698ac None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.243996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:55.279583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9383b21-ce75-4b7b-b5fd-db952e36d95d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.306122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:55.321716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d7f1b-42ec-4e12-83bc-58dc27e4a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.322589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d7f1b-42ec-4e12-83bc-58dc27e4a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.322876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d7f1b-42ec-4e12-83bc-58dc27e4a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.323152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d7f1b-42ec-4e12-83bc-58dc27e4a0c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.335455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d7f1b-42ec-4e12-83bc-58dc27e4a0c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.337131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d7f1b-42ec-4e12-83bc-58dc27e4a0c6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.349862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10005486-70f8-4ef7-b5e2-5b0ec28698ac None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-0a4d7f1b-42ec-4e12-83bc-58dc27e4a0c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:55.359598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:55.360711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:55.361817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:55.361817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:55.361817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:55.361817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:55.367487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 398/797] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:55.367487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754205-b58e-4527-8889-908bb99759c7 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.367487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754205-b58e-4527-8889-908bb99759c7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.367487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754205-b58e-4527-8889-908bb99759c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.387396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a9754205-b58e-4527-8889-908bb99759c7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:55.389640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a9754205-b58e-4527-8889-908bb99759c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:55.406272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.407487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.408398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.408876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.410212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:55.410958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.413030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.442611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 400/798] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:55.443631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.477799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 399/799] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:55.479995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9754205-b58e-4527-8889-908bb99759c7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.575363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:55.595958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6402f-5213-4919-9543-39bb068f079b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.618847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9754205-b58e-4527-8889-908bb99759c7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.621269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:55.634778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.636562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.637068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.637564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.638634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:55.639254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.640997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.661082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 401/800] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:55.662512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.664634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.665459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.665745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.666022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.680547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.682600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:55.705413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:55.706548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:55.707432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:55.707432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:55.707432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:55.707432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:55.710074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 400/801] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:55.711200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.787149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35154402-26f8-459f-a96b-e0aaa5b8e064 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.831736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7f82d-b91b-483e-8b81-fc31523e54e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.832552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7f82d-b91b-483e-8b81-fc31523e54e2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.832836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7f82d-b91b-483e-8b81-fc31523e54e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.833113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7f82d-b91b-483e-8b81-fc31523e54e2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.834035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7f82d-b91b-483e-8b81-fc31523e54e2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.835455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7f82d-b91b-483e-8b81-fc31523e54e2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.837772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2633267-0c15-4c59-9a75-eaaee344a88c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-c3e7f82d-b91b-483e-8b81-fc31523e54e2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:55.848427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:55.849630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:55.850536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 402/802] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:55.851375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793152ec-e699-423b-a97e-5ca5e408c869 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.851454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793152ec-e699-423b-a97e-5ca5e408c869 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.851726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793152ec-e699-423b-a97e-5ca5e408c869 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.858209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 403/803] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:55.858209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793152ec-e699-423b-a97e-5ca5e408c869 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.866433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:55.900510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:55.901615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:55.901803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:55.902050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:55.909502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:55.911004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:55.929388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:55.930369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:55.931265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 401/804] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:55.931762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:55.949250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:55.987561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793152ec-e699-423b-a97e-5ca5e408c869 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.025552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449a0d-a953-4392-aece-607b27ae4bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.026748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449a0d-a953-4392-aece-607b27ae4bd7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.027377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449a0d-a953-4392-aece-607b27ae4bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.027377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449a0d-a953-4392-aece-607b27ae4bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.028028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449a0d-a953-4392-aece-607b27ae4bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:56.028337 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449a0d-a953-4392-aece-607b27ae4bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.029753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449a0d-a953-4392-aece-607b27ae4bd7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.042039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 404/805] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:56.047997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bfd3-9b82-43d9-b6dc-aa6ee71843b9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.048279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bfd3-9b82-43d9-b6dc-aa6ee71843b9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.048858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bfd3-9b82-43d9-b6dc-aa6ee71843b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.051263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73f7ee-80dc-458a-8928-24ebe98c417b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.065721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6395bfd3-9b82-43d9-b6dc-aa6ee71843b9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:56.066962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6395bfd3-9b82-43d9-b6dc-aa6ee71843b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:56.089666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.090410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.090718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.091001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.091715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:56.092030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.093413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.098374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 402/806] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Jun 15 14:35:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:56.099427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.137502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:56.158333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 405/807] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:56.159280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395bfd3-9b82-43d9-b6dc-aa6ee71843b9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.184295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:56.206673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b1267-f995-41c4-acae-22e85e1e60e0 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.250576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.251905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.252269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.252534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.253349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:56.253661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.255147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.269224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 403/808] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:56.270372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.299205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395bfd3-9b82-43d9-b6dc-aa6ee71843b9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.340408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.341388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.341501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.341775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.349769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.351321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:56.367263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:56.368231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:56.369236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 406/809] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:56.369731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.389462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ad3c0-c715-4780-a0a0-fbf164a9ffa1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.425444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.426109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.426429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.426704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.427435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:56.427743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.428932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:56.429123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.434932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 404/810] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Jun 15 14:35:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:56.435646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.441849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:56.475368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73d517-fa04-4cd6-a7e8-b14d52d213da None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.510503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.511306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.511588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.511865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.524098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:56.531922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:56.533000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 407/811] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:56.534035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.550842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8b688-5ae5-4730-9e67-86cf106d9270 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.585832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.586759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.587034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.587303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/0b3a2fb853764be199cc6ff0dfd18064/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.589626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=0b3a2fb853764be199cc6ff0dfd18064, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.591107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.637961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148b514-404e-4f7f-a65b-8109af57b3c4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.653603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:56.672010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da9f7e-eca5-4b0d-9a5e-6ace103b0c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.672010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da9f7e-eca5-4b0d-9a5e-6ace103b0c04 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.672010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da9f7e-eca5-4b0d-9a5e-6ace103b0c04 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.672354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da9f7e-eca5-4b0d-9a5e-6ace103b0c04 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.673052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da9f7e-eca5-4b0d-9a5e-6ace103b0c04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:56.673363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da9f7e-eca5-4b0d-9a5e-6ace103b0c04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.674841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da9f7e-eca5-4b0d-9a5e-6ace103b0c04 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.687131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 408/812] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:56.688782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4467-e531-4fc5-80d7-f3b2a0509718 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.688916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4467-e531-4fc5-80d7-f3b2a0509718 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.689234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea4467-e531-4fc5-80d7-f3b2a0509718 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.691972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 409/813] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:56.691972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea4467-e531-4fc5-80d7-f3b2a0509718 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.698015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 405/814] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:56 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/0b3a2fb853764be199cc6ff0dfd18064/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:56.698888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.779398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:56.810224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea4467-e531-4fc5-80d7-f3b2a0509718 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.828092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d754f1c-7f54-46a4-98da-ce4fbc1e5686 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:56.851001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2eeb9-280f-4c2b-a6e3-c28409af1050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.851879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2eeb9-280f-4c2b-a6e3-c28409af1050 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.852316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2eeb9-280f-4c2b-a6e3-c28409af1050 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.852684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2eeb9-280f-4c2b-a6e3-c28409af1050 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.853495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2eeb9-280f-4c2b-a6e3-c28409af1050 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.855590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2eeb9-280f-4c2b-a6e3-c28409af1050 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.868370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:56.869054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.869511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.869895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.870683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:56.870998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:56.872461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:56.881940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 406/815] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:56.882805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:56.960246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 410/816] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:35:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:56.961867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaefeb8-b347-4c59-aa8d-4aeebf5a5168 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:56.962143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaefeb8-b347-4c59-aa8d-4aeebf5a5168 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:56.962464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaefeb8-b347-4c59-aa8d-4aeebf5a5168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:56.981329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-fbaefeb8-b347-4c59-aa8d-4aeebf5a5168 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:56.982735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fbaefeb8-b347-4c59-aa8d-4aeebf5a5168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:57.017688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c41e-f668-4ecf-b331-4d0c61925d2a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.026095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:57.054937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:57.077528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.078559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.078814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.079091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/fd1b940712c1481ba53ae1b45c68c347/roles/44450886f4e74405a0d654944c8740d4` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.081462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 411/817] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:57.082632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaefeb8-b347-4c59-aa8d-4aeebf5a5168 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.100521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=fd1b940712c1481ba53ae1b45c68c347, role_id=44450886f4e74405a0d654944c8740d4)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.102787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.177573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 407/818] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:35:56 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/fd1b940712c1481ba53ae1b45c68c347/roles/44450886f4e74405a0d654944c8740d4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:35:57.177963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.220125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaefeb8-b347-4c59-aa8d-4aeebf5a5168 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.274610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.276681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.277344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.277838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.295588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.297490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:57.327321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:57.328295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:57.329168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:57.329168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:57.329168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:57.329168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:57.329168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 412/819] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:57.331285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c5b0-80c6-410a-827a-ea07c68d555b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.334652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.374093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:57.394422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.395240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.395521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.395799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] PATH_INFO: `/v3/services/network` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.396466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.398436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:57.418416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:57.419475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:57.420993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 408/820] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:57.427205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.480977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:57.527506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb593d-853b-46a7-9477-17139fee8d55 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.584485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def126c3-99a0-4916-996c-1b1f805f3137 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.610287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.610958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.611240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.611514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.612244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:57.612551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.613953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.641769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 413/821] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:57.646963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.654746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.655414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.655694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.655967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.656673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] RBAC: Adding query filter params (name=network) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:57.656981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.658634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.673523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 409/822] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:57.679207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.761024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:57.794482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3595fa-cf6c-4eb5-8665-9aeccc05eed5 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.808175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:57.812700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89f05b-277d-4e85-84eb-9debd1e98912 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:57.852149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.852816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.853110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.853434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.855498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] RBAC: Adding query filter params (type=network) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:57.855630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:57.855834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.856335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:57.856621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:57.856900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:57.857188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.863472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 410/823] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:35:57.870888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:57.872387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:57.873781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:35:57.893425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:57.894443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:35:57.895374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:57.902103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 414/824] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:57.909798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:58.077139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8392767-1002-4c96-8a00-13120e97bd93 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:58.082431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4e90-b965-4655-b49e-068f37b67d8a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:58.145092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:58.146069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:58.146994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.147923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.148418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.148907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.149384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.149856 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.151455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:58.152749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:58.153284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:58.154956 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:58.156620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:58.160737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:58.180080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 415/825] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:35:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:58.186158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:58.272087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 411/826] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:35:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:35:58.303994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a51c3-ed24-4f25-bee9-da8148b071fc None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:58.367164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:58.368548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.369193 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.369754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.448722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:58.465350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:58.468200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:58.659286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 416/827] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:35:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:35:58.664749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:58.701891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:58.796780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cad7e-9675-4b0b-af49-109fded9e2c4 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:58.836965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:58.837878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:58.838384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:58.838827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:58.849965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:58.851475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:35:58.874285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:58.875213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:58.876118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:35:58.876118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:35:58.876118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:35:58.876118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:58.876118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 412/828] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:58.881707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:58.986748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:58.998964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddd43b-8fa2-41e9-9b5a-30a69b74c990 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:59.155307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762137b4-32a1-47fb-9a2a-b337d9bfeebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:59.155980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762137b4-32a1-47fb-9a2a-b337d9bfeebb None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.156265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762137b4-32a1-47fb-9a2a-b337d9bfeebb None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.156543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762137b4-32a1-47fb-9a2a-b337d9bfeebb None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.157274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762137b4-32a1-47fb-9a2a-b337d9bfeebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:59.157584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762137b4-32a1-47fb-9a2a-b337d9bfeebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:59.159004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762137b4-32a1-47fb-9a2a-b337d9bfeebb None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:59.171363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 417/829] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:59.176881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:59.257672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:59.291560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd6fcc-2c9e-4696-ba48-3bd474f45c40 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:59.307844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f448-822c-4268-bc0a-3f24a6e4c85c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.308180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f448-822c-4268-bc0a-3f24a6e4c85c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.308413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f448-822c-4268-bc0a-3f24a6e4c85c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.310119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 418/830] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:35:59.319021 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0b59-7e94-493f-abe5-5ef3487cd196 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.319316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0b59-7e94-493f-abe5-5ef3487cd196 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.321101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0b59-7e94-493f-abe5-5ef3487cd196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.336990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3b0a0b59-7e94-493f-abe5-5ef3487cd196 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:59.339079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3b0a0b59-7e94-493f-abe5-5ef3487cd196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:59.340443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:59.341485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.341774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.342051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.351518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:59.353322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:35:59.377206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:59.379012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:35:59.380134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 413/831] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:35:59.385497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:59.422734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 419/832] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:35:59.446892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a0b59-7e94-493f-abe5-5ef3487cd196 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:59.511475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6cd7-ebf4-4b8e-ae6a-110bf46f10cf None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:59.530544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:59.543231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:59.549359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:59.550219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.550368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.550654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.551446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:59.551766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:59.553190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:59.555012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a0b59-7e94-493f-abe5-5ef3487cd196 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:59.570486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 414/833] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:59.572927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:59.594752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:59.595843 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.596317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.596743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.610776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:35:59.619414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:35:59.620458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:35:59.621385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:35:59.628450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 420/834] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:35:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:35:59.629676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:59.689377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf7d6c-2c9d-495c-8ac6-0ae405a87082 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:59.725846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c9141-c624-407c-8348-3e6188dc3d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:59.726605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9141-c624-407c-8348-3e6188dc3d2f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.726886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9141-c624-407c-8348-3e6188dc3d2f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.727162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9141-c624-407c-8348-3e6188dc3d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.728046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9141-c624-407c-8348-3e6188dc3d2f None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=ac6a657099c14f598d5becd5ab72ee84) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:59.728873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9141-c624-407c-8348-3e6188dc3d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:59.730388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9141-c624-407c-8348-3e6188dc3d2f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:59.738668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 415/835] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:35:59 2022] GET /identity/v3/role_assignments?user.id=ac6a657099c14f598d5becd5ab72ee84&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:59.741823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361866e-6762-414b-a210-65bf4ee268dd None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:35:59.745596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88877d9a-90b0-429e-a7db-6a0d25405aac None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.746281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88877d9a-90b0-429e-a7db-6a0d25405aac None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.746768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88877d9a-90b0-429e-a7db-6a0d25405aac None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.748841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 416/836] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:59.754853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f69b-0921-49de-944b-6881e7505f34 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.755173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f69b-0921-49de-944b-6881e7505f34 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.755521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f69b-0921-49de-944b-6881e7505f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.775655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3a08f69b-0921-49de-944b-6881e7505f34 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:59.776828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3a08f69b-0921-49de-944b-6881e7505f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:59.787135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:35:59.788199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.788725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.789206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.790208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:35:59.790725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:35:59.792926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:35:59.799539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:59.804654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 421/837] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:35:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:35:59.811288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:35:59.854622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:35:59.882237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 417/838] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:35:59.892763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c358cb9-3d14-49b9-817c-f16668c7b903 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.893094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c358cb9-3d14-49b9-817c-f16668c7b903 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.893324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c358cb9-3d14-49b9-817c-f16668c7b903 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.895494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 418/839] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:35:59.901409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb839b-2ef5-4eef-a744-dfcc844538ae None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:35:59.901726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb839b-2ef5-4eef-a744-dfcc844538ae None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:35:59.902019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb839b-2ef5-4eef-a744-dfcc844538ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:35:59.920201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b6cb839b-2ef5-4eef-a744-dfcc844538ae None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:35:59.921435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b6cb839b-2ef5-4eef-a744-dfcc844538ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:35:59.961118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2548fe-6df2-4559-ba3b-c6705a1172c7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.014399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 419/840] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:35:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:00.014399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb839b-2ef5-4eef-a744-dfcc844538ae None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.023626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.025339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.026452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.027206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.040567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.042504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:00.062439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:00.063417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:00.064315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:00.064315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:00.064315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:00.064315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:00.064315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 422/841] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:00.064613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.129088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb839b-2ef5-4eef-a744-dfcc844538ae None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.130379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:00.149306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:00.174621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.175496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.176026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.176309 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.176985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.178425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.187183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a2eb0-5010-4709-8b62-3c499e697977 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:00.192766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:00.200547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:00.201600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 420/842] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:00.202707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.257209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.258046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.258314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.258591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.270792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:00.278849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:00.279770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:00.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 423/843] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:00.281285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.311579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144c3c6-7511-4a9f-8b65-a08a81728c23 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.346914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.347687 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.347972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.348256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.348985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:00.349286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.350712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.364307 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 421/844] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:00.365162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.400246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb856b8-0010-497c-aba4-28930e225990 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.412083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:00.417201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:00.437863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.438586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.438854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.439128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.439857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:00.440151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.441541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.446205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 424/845] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:00.447560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.488990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dccc10-8dc5-4f2f-90b9-a923e950fbf1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.532658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.533350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.533635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.533914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.534708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:00.535020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.536879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.546437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 422/846] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:00.547247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.567967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124d873-530d-46d0-a6b5-3a029d83e328 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.606066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.606900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.607182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.607455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.616102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.617629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:00.633144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:00.634156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 425/847] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:00.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.673110 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:00.686252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:00.687001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334a0b8-99e7-4826-a7e3-85bc240d2ec6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.729153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.730369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.730980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.731352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.732074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:00.732387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.733899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.750201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 423/848] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:00.750201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.765008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556c767-b9fe-49fc-bd6c-383e8fb496a4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.805451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.806953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.807265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.807606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.820672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.823097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:00.843812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:00.844967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:00.846086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:00.846086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:00.846086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:00.846086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:00.849105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 426/849] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:00.853934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.885546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7c08f-0307-44c4-9ad3-d8c02ef8cae1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:00.932596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:00.933533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:00.934027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:00.934467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:00.935389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:00.935707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:00.937228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:00.959947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 424/850] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:00.966932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:00.982820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.007010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed363c3-ffa7-4cfc-b628-435d67d960eb None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.008342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.063606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.064767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.066044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.066912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.067862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:01.068478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.070087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.091066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 427/851] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:01.092659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:01.105149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30ef79-bb2d-4956-ac95-ee434cea8ed3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.141904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.142681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.143038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.143395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.205581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219231f5-3fb2-4fbe-aec0-0965f20c4c6a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.215282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.216759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.241849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.245926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.247660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.248094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.248687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.258568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.261089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:01.281528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:01.288663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 428/852] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:01.289582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:01.305185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.358441 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 425/853] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:01.364356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:01.452841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69623-e801-49b1-8a93-d9b171031848 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.469509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4d7e-e1e0-4096-9d81-6933d7ea8ce7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.493210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.505975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.506810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.507089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.508551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.519003 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.519232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.520157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.520520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.520954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.521333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.522457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:01.522972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.526397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:01.544904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:01.545863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:01.546786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:01.546786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:01.546786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:01.546786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:01.552679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 429/854] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:01.553903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:01.554278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 426/855] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:01.559563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:01.634023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.669289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a929-e1f2-4558-b6fc-22dde9bde079 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.674717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d4b58-8169-444c-a4e9-465efd2b4860 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.706805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.707906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.708701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.709178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.710160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:01.710899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.712676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.714716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.715476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.715763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.716057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.727659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 427/856] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:01.735241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:01.771979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.808164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.809665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.861375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1eed-457c-4b1d-8f9b-70f3e9968b45 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:01.895217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:01.902905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:01.903800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:01.908832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:01.908832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:01.913620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:01.916359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:01.936228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:01.937217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:01.938099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:01.942718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 428/857] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:01.948752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:01.949330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 430/858] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:01.953343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.087386 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb81ddb-57d3-41ef-886d-8e5a4633f72f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.094947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70395dd1-57a6-4f57-b26d-b0d7608b46f3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.098624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:02.136580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58950fc-786b-4ef0-952a-ef5c37380cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.137246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58950fc-786b-4ef0-952a-ef5c37380cdb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.137531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58950fc-786b-4ef0-952a-ef5c37380cdb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.137806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58950fc-786b-4ef0-952a-ef5c37380cdb None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.138692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58950fc-786b-4ef0-952a-ef5c37380cdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:02.138982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58950fc-786b-4ef0-952a-ef5c37380cdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.140281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba24da-ae2b-4a42-a6a3-7db174d52a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.140405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58950fc-786b-4ef0-952a-ef5c37380cdb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.141359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba24da-ae2b-4a42-a6a3-7db174d52a0e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.141817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba24da-ae2b-4a42-a6a3-7db174d52a0e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.142377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba24da-ae2b-4a42-a6a3-7db174d52a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.154347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 429/859] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:02.156067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b28d-c8a8-4fe8-9333-f4736f3bb876 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.156351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b28d-c8a8-4fe8-9333-f4736f3bb876 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.156624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b28d-c8a8-4fe8-9333-f4736f3bb876 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.158055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 430/860] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:02.160683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16b28d-c8a8-4fe8-9333-f4736f3bb876 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.161081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba24da-ae2b-4a42-a6a3-7db174d52a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.162889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba24da-ae2b-4a42-a6a3-7db174d52a0e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-ebba24da-ae2b-4a42-a6a3-7db174d52a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:02.183019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:02.184124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:02.185240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:02.185240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:02.185240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:02.185240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:02.186832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 431/861] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:02.188366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d894e-3b06-446e-ab59-fcf4a019b226 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.188800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d894e-3b06-446e-ab59-fcf4a019b226 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.189237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d894e-3b06-446e-ab59-fcf4a019b226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.208967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-728d894e-3b06-446e-ab59-fcf4a019b226 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:02.210206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-728d894e-3b06-446e-ab59-fcf4a019b226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:02.224938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:02.283241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 432/862] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:02.285222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16b28d-c8a8-4fe8-9333-f4736f3bb876 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.298583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d894e-3b06-446e-ab59-fcf4a019b226 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.330546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.331333 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.331620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.331897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.332733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=0b3a2fb853764be199cc6ff0dfd18064) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:02.333095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.334887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.354210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 431/863] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/role_assignments?user.id=0b3a2fb853764be199cc6ff0dfd18064&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:02.354210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.382038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:02.438161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d894e-3b06-446e-ab59-fcf4a019b226 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.481522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.482401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.482686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.482961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.483713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:02.484024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.485453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.502783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 433/864] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:02.508507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.524884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457aae1-8f16-4b61-833c-7aa82ec825a8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.527620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:02.589198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.590024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.590342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.590624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.592094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.593654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:02.609360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:02.610574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:02.611610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 432/865] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:02.618628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.667741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d95a3f-5c57-4b51-9926-1bc283eb0f47 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.709821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.710890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.711315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.711750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.719846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.721575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:02.738823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:02.739772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:02.740676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:02.743623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 434/866] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:02.746061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:02.749311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.760836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b82c-dcbc-48e5-9a80-b47e2f43a9b5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.802409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.803044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.803334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.803614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.804393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:02.804709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.806889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.813057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 433/867] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:02.817892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.836340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:02.879079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6a796-a55a-4563-8940-3817f9d4e447 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.919067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.919746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.920032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.920311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.921111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:02.921421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.922921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:02.940946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 435/868] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:02.941891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:02.945014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74142155-5e37-4eee-becb-b18799a7f89b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:02.990858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd7c62-692d-437d-98e3-5109d867748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:02.991468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd7c62-692d-437d-98e3-5109d867748b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:02.991764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd7c62-692d-437d-98e3-5109d867748b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:02.992028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd7c62-692d-437d-98e3-5109d867748b None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:02.992766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd7c62-692d-437d-98e3-5109d867748b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:02.993075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd7c62-692d-437d-98e3-5109d867748b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:02.995194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd7c62-692d-437d-98e3-5109d867748b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:03.001421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 434/869] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:03.045907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37763a27-e0b7-4f84-82b3-0e0686d584fd None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:03.046247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37763a27-e0b7-4f84-82b3-0e0686d584fd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:03.046496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37763a27-e0b7-4f84-82b3-0e0686d584fd None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:03.048140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 435/870] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:03.053645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc8951-bcd2-433b-9c96-8124144c8a4b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:03.053957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc8951-bcd2-433b-9c96-8124144c8a4b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:03.054214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc8951-bcd2-433b-9c96-8124144c8a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:03.083782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-21dc8951-bcd2-433b-9c96-8124144c8a4b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:03.085051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-21dc8951-bcd2-433b-9c96-8124144c8a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:03.085854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edf394-2afc-4c79-84e1-2e2916af2f92 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:03.091128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:03.122398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:03.132265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:03.133340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:03.133862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:03.134874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:03.136169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=fd1b940712c1481ba53ae1b45c68c347) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:03.136727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:03.138467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:03.145673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 436/871] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:02 2022] GET /identity/v3/role_assignments?user.id=fd1b940712c1481ba53ae1b45c68c347&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:03.199251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 436/872] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:03.223453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:03.362460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed30c1a-895d-4db4-9f0f-ba1315899f81 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:03.401176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421eeea-7ad8-4d1d-aa1d-e67a5635138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:03.401870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421eeea-7ad8-4d1d-aa1d-e67a5635138e None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:03.402155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421eeea-7ad8-4d1d-aa1d-e67a5635138e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:03.402457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421eeea-7ad8-4d1d-aa1d-e67a5635138e None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:03.403203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421eeea-7ad8-4d1d-aa1d-e67a5635138e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:03.404609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421eeea-7ad8-4d1d-aa1d-e67a5635138e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:03.481117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 437/873] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:36:03 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:36:04.767703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cf4ff-61dd-416d-92a9-801e12ff3dd6 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:04.768721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cf4ff-61dd-416d-92a9-801e12ff3dd6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:04.769215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cf4ff-61dd-416d-92a9-801e12ff3dd6 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:04.771111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 437/874] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:04.776891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f80a8-628a-48d7-a857-6dbd02cc4e97 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:04.777790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f80a8-628a-48d7-a857-6dbd02cc4e97 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:04.778333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f80a8-628a-48d7-a857-6dbd02cc4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:04.796101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3c2f80a8-628a-48d7-a857-6dbd02cc4e97 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:04.797596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3c2f80a8-628a-48d7-a857-6dbd02cc4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:04.855076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:04.890226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 438/875] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:04.913451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cf4ff-61dd-416d-92a9-801e12ff3dd6 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:05.038931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cf4ff-61dd-416d-92a9-801e12ff3dd6 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:05.085694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:05.086959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.088096 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.088592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.090753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:05.092403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:05.109061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:05.110123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:05.110985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:05.113875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 438/876] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:05.120241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f80a8-628a-48d7-a857-6dbd02cc4e97 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:05.231420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f80a8-628a-48d7-a857-6dbd02cc4e97 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:05.272505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ecfec-71a6-4f34-bc06-ceaea13f3b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:05.273179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ecfec-71a6-4f34-bc06-ceaea13f3b26 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.273484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ecfec-71a6-4f34-bc06-ceaea13f3b26 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.273732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ecfec-71a6-4f34-bc06-ceaea13f3b26 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.274462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ecfec-71a6-4f34-bc06-ceaea13f3b26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:05.275355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ecfec-71a6-4f34-bc06-ceaea13f3b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:05.277080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ecfec-71a6-4f34-bc06-ceaea13f3b26 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:05.282627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 439/877] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:05.288299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:05.293476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:05.309090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5c352-91aa-49f4-8570-9362bfef1b9c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.309407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5c352-91aa-49f4-8570-9362bfef1b9c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.309645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5c352-91aa-49f4-8570-9362bfef1b9c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.311453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 440/878] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:05.316821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4d11f-ee91-48ba-a66d-3807729fc038 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.317171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4d11f-ee91-48ba-a66d-3807729fc038 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.317382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4d11f-ee91-48ba-a66d-3807729fc038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.335673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-39f4d11f-ee91-48ba-a66d-3807729fc038 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:05.336693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-39f4d11f-ee91-48ba-a66d-3807729fc038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:05.402590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:05.411761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c4d5-ac56-40e6-a0ac-56f5bb40b3ab None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:05.442791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 441/879] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:05.459089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd4065-ac97-4841-bc64-f245d46a8baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:05.459795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd4065-ac97-4841-bc64-f245d46a8baf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.460123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd4065-ac97-4841-bc64-f245d46a8baf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.460407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd4065-ac97-4841-bc64-f245d46a8baf None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.461156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd4065-ac97-4841-bc64-f245d46a8baf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:05.461468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd4065-ac97-4841-bc64-f245d46a8baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:05.463259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd4065-ac97-4841-bc64-f245d46a8baf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:05.469086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 439/880] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:05.475383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4d11f-ee91-48ba-a66d-3807729fc038 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:05.514216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c9e14-403c-4be2-b1b2-5a5f11970bac None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.514613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c9e14-403c-4be2-b1b2-5a5f11970bac None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.514764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c9e14-403c-4be2-b1b2-5a5f11970bac None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.516562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 440/881] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:05.522685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e0277-702e-4905-8da8-f63022a1c1e3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.522685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e0277-702e-4905-8da8-f63022a1c1e3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.522685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e0277-702e-4905-8da8-f63022a1c1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.544240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-fd8e0277-702e-4905-8da8-f63022a1c1e3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:05.545755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fd8e0277-702e-4905-8da8-f63022a1c1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:05.619682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4d11f-ee91-48ba-a66d-3807729fc038 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:05.627598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:05.638492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 441/882] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:05.660383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e0277-702e-4905-8da8-f63022a1c1e3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:05.666237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b977955-6526-42a1-9924-da4c1f9416b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:05.667992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b977955-6526-42a1-9924-da4c1f9416b0 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.667992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b977955-6526-42a1-9924-da4c1f9416b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.667992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b977955-6526-42a1-9924-da4c1f9416b0 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.668379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b977955-6526-42a1-9924-da4c1f9416b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:05.669723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b977955-6526-42a1-9924-da4c1f9416b0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:05.763327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 442/883] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:36:05 2022] POST /identity/v3/services => generated 269 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:05.794337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e0277-702e-4905-8da8-f63022a1c1e3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:05.849009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:05.849871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.850153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.850460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.855079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a547ef-37e8-400b-a892-ae7690a66e55 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.855595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a547ef-37e8-400b-a892-ae7690a66e55 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.855670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a547ef-37e8-400b-a892-ae7690a66e55 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.857340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 443/884] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:05.863670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9cb23-3e8c-4309-a604-01bb04e3e31a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:05.864009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9cb23-3e8c-4309-a604-01bb04e3e31a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:05.864228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9cb23-3e8c-4309-a604-01bb04e3e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:05.868291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:05.880158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:05.881174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:05.882026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 442/885] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Jun 15 14:36:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:05.886202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-bae9cb23-3e8c-4309-a604-01bb04e3e31a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:05.886485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bae9cb23-3e8c-4309-a604-01bb04e3e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:05.890660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:05.928555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:05.969169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:05.990217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 444/886] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:06.014094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9cb23-3e8c-4309-a604-01bb04e3e31a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.034553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d084ab-7f15-432e-8284-6e60c2734ba3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.074917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.075758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.076137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.076508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.077276 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:06.077581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.079138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.088900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 443/887] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Jun 15 14:36:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:06.096230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.153241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9cb23-3e8c-4309-a604-01bb04e3e31a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.193466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.194302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.194579 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.194848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.195507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.197481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:06.210955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:06.212103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:06.213658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:06.218045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 445/888] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:06.219050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.227639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720144a-55f4-4f7c-8b3c-176fcdc3cdda None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.249656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:06.250264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:06.276480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.277542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.278003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.278822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.294427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.296763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:06.319861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:06.321162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:06.322830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:06.322830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:06.322830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:06.322830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:06.328923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 444/889] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:06.330785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.343301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fa146-931e-43ef-a747-5a8b4c616e03 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.384008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.385045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.385297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.385902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.386563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:06.386842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.388254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.398662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 446/890] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:06.399791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.490138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab938d20-5e5e-439d-90c5-d8b68b1a83b7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.520814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:06.537099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52d7ed-5ab0-4975-bcc6-f46e7c9555d7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.546921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.547598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.547879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.550465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.551555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:06.553301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.557482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.573392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 445/891] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:06.578933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.591082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.591771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.592057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.592337 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.593616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:06.594184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.596098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.608911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 447/892] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:06.610253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.618254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:06.713560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71da71-25f0-4562-b94c-e4be40b916bd None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.748111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f0340-b740-414d-9d2e-3d9051ed7fdf None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.754493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.755373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.755714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.755992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.764474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.765945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:06.791698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:06.792720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:06.793640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:06.796044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.796959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.797435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.797901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.798684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 446/893] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:06.799583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.800333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] RBAC: Adding query filter params (interface=public, service_id=259cd5b22dc7421aa5a0b5dea102157d, region_id=RegionOne) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:06.801178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.802900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.811742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 448/894] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/endpoints?service_id=259cd5b22dc7421aa5a0b5dea102157d&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:06.818004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:06.839809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:06.905811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:06.919267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a012846-80c3-464e-90c4-d665ec4435e3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.930150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0a70-0b0e-470c-b039-806e3151b183 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:06.958578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.959593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.960155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.960604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.961724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:06.962306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.964265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.969312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:06.969991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:06.970329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:06.970770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:06.971470 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:06.972850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:06.979971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 449/895] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:36:06.986743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 447/896] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:06.996152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:07.097741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:07.119945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0bfef-94d0-43b1-a159-aeadd638878e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:07.162992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:07.163750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.164206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.164547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.238539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:07.240148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:07.330875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:07.351902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 450/897] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:06 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:07.357374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:07.469301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72727492-db16-452a-8e3f-f6c2657f24dc None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:07.514533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896459-c786-4039-8e5c-d41477712881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:07.515817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896459-c786-4039-8e5c-d41477712881 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.516619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896459-c786-4039-8e5c-d41477712881 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.517435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896459-c786-4039-8e5c-d41477712881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.534193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896459-c786-4039-8e5c-d41477712881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:07.534193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896459-c786-4039-8e5c-d41477712881 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:07.542288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-16896459-c786-4039-8e5c-d41477712881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:07.557518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:07.558773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:07.559668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:07.559668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:07.559668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:07.559668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:07.559668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 448/898] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:07.567212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:07.694301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f7155-3c4a-48b0-a8b1-ac5fc22aee83 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:07.735220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc52e7-29d2-4dff-a901-979bf2423119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:07.735899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc52e7-29d2-4dff-a901-979bf2423119 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.736178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc52e7-29d2-4dff-a901-979bf2423119 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.736449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc52e7-29d2-4dff-a901-979bf2423119 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.737181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc52e7-29d2-4dff-a901-979bf2423119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:07.737483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc52e7-29d2-4dff-a901-979bf2423119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:07.738896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc52e7-29d2-4dff-a901-979bf2423119 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:07.752318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 451/899] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:07.757940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896459-c786-4039-8e5c-d41477712881 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:07.783618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b3060-30af-47f2-adb6-679c7dfee17c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.783954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b3060-30af-47f2-adb6-679c7dfee17c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.784172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b3060-30af-47f2-adb6-679c7dfee17c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.786226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 452/900] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:07.791634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebb8e9-3e84-4489-b6fd-d48a8da672b4 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.792117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebb8e9-3e84-4489-b6fd-d48a8da672b4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.792553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebb8e9-3e84-4489-b6fd-d48a8da672b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.811357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-08ebb8e9-3e84-4489-b6fd-d48a8da672b4 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:07.812782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-08ebb8e9-3e84-4489-b6fd-d48a8da672b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:07.855134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:07.878266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896459-c786-4039-8e5c-d41477712881 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:07.890806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 453/901] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:07.914893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebb8e9-3e84-4489-b6fd-d48a8da672b4 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:07.923305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:07.924585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:07.925103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:07.925581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:07.934160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:07.935990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:07.952383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:07.953296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:07.954190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:07.961568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 449/902] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:07.962895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:08.037421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:08.039218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebb8e9-3e84-4489-b6fd-d48a8da672b4 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:08.087643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621840b-91c4-4fc0-bec8-7b7cdd7e0ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:08.088960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621840b-91c4-4fc0-bec8-7b7cdd7e0ce8 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.089251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621840b-91c4-4fc0-bec8-7b7cdd7e0ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.089528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621840b-91c4-4fc0-bec8-7b7cdd7e0ce8 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.090862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621840b-91c4-4fc0-bec8-7b7cdd7e0ce8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:08.092302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621840b-91c4-4fc0-bec8-7b7cdd7e0ce8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:08.092819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e71a5f-c213-4c94-b351-8c0e4aa3fbf7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:08.137088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:08.137768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.138052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.138364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.139171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:08.139485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:08.140948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:08.156307 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 450/903] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:08.162228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:08.169121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 454/904] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:36:07 2022] POST /identity/v3/services => generated 241 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:36:08.170771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932b428-6ccf-44cc-a711-95c8e448883e None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.171242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932b428-6ccf-44cc-a711-95c8e448883e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.171696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932b428-6ccf-44cc-a711-95c8e448883e None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.173865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 455/905] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:08.180011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca4adc-9dec-492d-bb0d-c36213747c0e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.180616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca4adc-9dec-492d-bb0d-c36213747c0e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.181394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca4adc-9dec-492d-bb0d-c36213747c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.207606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-59ca4adc-9dec-492d-bb0d-c36213747c0e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:08.209608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-59ca4adc-9dec-492d-bb0d-c36213747c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:08.253562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:08.304787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb3424-cb00-48bb-a669-6a76ce7d59f2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:08.325379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:08.329153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 456/906] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:08.350363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:08.351458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.352756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca4adc-9dec-492d-bb0d-c36213747c0e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:08.354681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.355213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.356315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] RBAC: Adding query filter params (role.id=30837f7a6e9442c2a8d667aed7aa1683, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=fd1b940712c1481ba53ae1b45c68c347) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:08.356877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:08.358716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:08.369701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 451/907] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:08 2022] GET /identity/v3/role_assignments?user.id=fd1b940712c1481ba53ae1b45c68c347&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=30837f7a6e9442c2a8d667aed7aa1683 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:08.482933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca4adc-9dec-492d-bb0d-c36213747c0e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:08.522469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:08.523630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.524063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.524481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.526197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:08.527988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:08.545562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:08.546778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:08.547818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:08.553851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 457/908] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Jun 15 14:36:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:08.558440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:08.694521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b49d16-7382-4dcf-b9aa-5e27b0529174 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:08.736839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:08.737809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.738323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.738777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.739680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:08.740167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:08.741747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:08.748330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 452/909] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:08.754148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:08.791043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:08.876907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670215e-143d-45ef-9820-0f2aac323c96 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:08.912658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:08.913344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:08.913628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:08.913907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:08.914657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:08.914966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:08.916621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:08.922569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 458/910] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:08.928447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:08.987793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:09.055513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69869b93-4e48-4bbc-b037-b2a27bd158be None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:09.096456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bd6e0-1002-48a6-b644-5cb3b4aecb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:09.097136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bd6e0-1002-48a6-b644-5cb3b4aecb26 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.097421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bd6e0-1002-48a6-b644-5cb3b4aecb26 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.097698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bd6e0-1002-48a6-b644-5cb3b4aecb26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.098543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bd6e0-1002-48a6-b644-5cb3b4aecb26 None admin] RBAC: Adding query filter params (interface=public, service_id=ccdc7e6c47274cadb4bc906c87d8cc43, region_id=RegionOne) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:09.098862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bd6e0-1002-48a6-b644-5cb3b4aecb26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:09.100252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bd6e0-1002-48a6-b644-5cb3b4aecb26 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:09.109529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 453/911] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:36:08 2022] GET /identity/v3/endpoints?service_id=ccdc7e6c47274cadb4bc906c87d8cc43&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:09.115197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:09.220301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:09.239015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed49aab-ec40-4757-9782-240173038b01 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:09.282828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22275121-3b30-4e50-ac33-c625b5713476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:09.283534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22275121-3b30-4e50-ac33-c625b5713476 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.283814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22275121-3b30-4e50-ac33-c625b5713476 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.284089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22275121-3b30-4e50-ac33-c625b5713476 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.284805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22275121-3b30-4e50-ac33-c625b5713476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:09.286287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22275121-3b30-4e50-ac33-c625b5713476 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:09.294925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 459/912] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:36:09 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:36:09.344634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab73ec-a751-4140-9ce2-796037f49ffd None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.344962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab73ec-a751-4140-9ce2-796037f49ffd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.345207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab73ec-a751-4140-9ce2-796037f49ffd None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.347019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 454/913] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:09.355218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623da96-478a-4593-8937-637657037ddd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.355690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623da96-478a-4593-8937-637657037ddd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.356122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623da96-478a-4593-8937-637657037ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.376991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3623da96-478a-4593-8937-637657037ddd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:09.378242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3623da96-478a-4593-8937-637657037ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:09.472766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 460/914] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:09.497692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab73ec-a751-4140-9ce2-796037f49ffd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:09.615412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab73ec-a751-4140-9ce2-796037f49ffd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:09.629814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:09.655358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:09.656167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.656567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.656907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.657570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:09.659105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:09.672045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:09.674328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 14:36:09.676248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:09.682222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 455/915] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:09.682560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623da96-478a-4593-8937-637657037ddd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:09.770656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:09.790748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623da96-478a-4593-8937-637657037ddd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:09.836121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:09.836797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:09.837230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:09.837647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:09.838577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:09.839103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:09.840855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:09.846543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 461/916] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:09.851999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:09.963338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291b92c-a41c-40da-a090-5b66fb66c58e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:10.000798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06020475-afe9-40cf-8ff5-2ba4f7cfc390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:10.001935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06020475-afe9-40cf-8ff5-2ba4f7cfc390 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.002498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06020475-afe9-40cf-8ff5-2ba4f7cfc390 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.002981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06020475-afe9-40cf-8ff5-2ba4f7cfc390 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.003936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06020475-afe9-40cf-8ff5-2ba4f7cfc390 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:10.004449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06020475-afe9-40cf-8ff5-2ba4f7cfc390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:10.006033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06020475-afe9-40cf-8ff5-2ba4f7cfc390 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:10.016755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 456/917] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Jun 15 14:36:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:10.018663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:10.155800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650f45b-fd8e-4e82-a5c6-55f1a4a9ada4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:10.197205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147b99d-5e37-4425-8bf7-7d4c97f46b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:10.198130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147b99d-5e37-4425-8bf7-7d4c97f46b85 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.198444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147b99d-5e37-4425-8bf7-7d4c97f46b85 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.198722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147b99d-5e37-4425-8bf7-7d4c97f46b85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.199493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147b99d-5e37-4425-8bf7-7d4c97f46b85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:10.200910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147b99d-5e37-4425-8bf7-7d4c97f46b85 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:10.281229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:10.292846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 462/918] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:36:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:10.584987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb432a2-bcac-4acd-9eae-8bd7f4d54ad1 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.585402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb432a2-bcac-4acd-9eae-8bd7f4d54ad1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.585573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb432a2-bcac-4acd-9eae-8bd7f4d54ad1 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.587915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 457/919] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:10.593840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84426fb-6659-42d8-9025-f071395a0276 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.594224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84426fb-6659-42d8-9025-f071395a0276 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.594428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84426fb-6659-42d8-9025-f071395a0276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.616688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a84426fb-6659-42d8-9025-f071395a0276 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:10.617929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a84426fb-6659-42d8-9025-f071395a0276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:10.709021 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 463/920] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:10.732558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb432a2-bcac-4acd-9eae-8bd7f4d54ad1 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:10.734961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd966826-c051-431a-add1-cc9ce6154b13 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.735203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd966826-c051-431a-add1-cc9ce6154b13 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.735476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd966826-c051-431a-add1-cc9ce6154b13 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.740836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 464/921] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:10.743709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e522b7-3d52-4589-9f76-ca2068864df7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.743994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e522b7-3d52-4589-9f76-ca2068864df7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.744266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e522b7-3d52-4589-9f76-ca2068864df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.764547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-50e522b7-3d52-4589-9f76-ca2068864df7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:10.765764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-50e522b7-3d52-4589-9f76-ca2068864df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:10.799115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:10.846552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 465/922] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:10.866246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb432a2-bcac-4acd-9eae-8bd7f4d54ad1 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:10.868038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e522b7-3d52-4589-9f76-ca2068864df7 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:10.919823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:10.920979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:10.921413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:10.921941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:10.922785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:10.924484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:10.937791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:10.939597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:10.940783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:10.941779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:10.949495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 458/923] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:10.950760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:10.989878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e522b7-3d52-4589-9f76-ca2068864df7 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.034032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.035132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.035595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.036034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.048419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:11.058246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:11.059465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:11.060428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:11.062966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 466/924] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Jun 15 14:36:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:11.069034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.075868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7174d-cd63-451c-872d-253de8ba98ab None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.078589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:11.130106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.130838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.131176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.131454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.132478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:11.132798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.134910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.142220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 459/925] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:11.147304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.206663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eaad4-f555-40f7-a865-94b044dc45fb None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.230737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:11.255424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.256653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.257592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.258070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.259444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:11.259976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.262321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.283207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f0488-d212-410a-b95a-fd772a98cc49 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.283409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 467/926] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:11.285602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.323891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.324498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.324782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.325057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.325770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:11.326080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.327594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.341867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 460/927] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:11.344199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.387270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:11.424356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedf344-573a-476b-b3a9-bf1014299238 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.466290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.467188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.467540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.467892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.480347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.482452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.494743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f4fc-5307-4ab0-ad99-526886477b9c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:11.503200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:11.504157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:11.505039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:11.505039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:11.505039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:11.505039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:11.509955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 468/928] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:11.511200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.547166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.547842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.548047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:11.548194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.548444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.549268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] RBAC: Adding query filter params (interface=public, service_id=90487d65674b4282aee85d5ef510abe1, region_id=RegionOne) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:11.549732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.551788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.561403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 461/929] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/endpoints?service_id=90487d65674b4282aee85d5ef510abe1&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:11.567123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.636971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04701767-8e28-4d77-ad5e-54ca05401dbd None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.676639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.677662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.678328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.678809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.679893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:11.680462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.682240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.702583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 469/930] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:11.703997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.711145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:11.720752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fffa92-cc24-4647-be42-1e0fbe481459 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.767803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95ffe0-9575-4aff-9807-86262c08334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.768459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ffe0-9575-4aff-9807-86262c08334e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.768745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ffe0-9575-4aff-9807-86262c08334e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.769022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95ffe0-9575-4aff-9807-86262c08334e None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.769724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95ffe0-9575-4aff-9807-86262c08334e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.771154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95ffe0-9575-4aff-9807-86262c08334e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.786677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 462/931] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/services => generated 1497 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:36:11.786677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703c87b-76fc-43f6-8c9d-76ead48c640f None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.786677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703c87b-76fc-43f6-8c9d-76ead48c640f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.786677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703c87b-76fc-43f6-8c9d-76ead48c640f None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.786677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 463/932] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:11.792517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550d03f-2473-4025-afd9-c8f0fef1c582 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.793848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550d03f-2473-4025-afd9-c8f0fef1c582 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.795498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550d03f-2473-4025-afd9-c8f0fef1c582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.826028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6550d03f-2473-4025-afd9-c8f0fef1c582 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:11.827791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6550d03f-2473-4025-afd9-c8f0fef1c582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:11.832433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:11.849925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e748424-fdb7-44b1-819a-548a138de684 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:11.904002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:11.905088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:11.905691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:11.906236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:11.916837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:11.919044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:11.939922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:11.940941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:11.941855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:11.949823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 464/934] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:11.950343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 470/934] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:11.955523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550d03f-2473-4025-afd9-c8f0fef1c582 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:11.968610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.084731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:12.100926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550d03f-2473-4025-afd9-c8f0fef1c582 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.109038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2249733-29de-46ba-99fa-e6f782ab5276 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.150106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.151304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.151689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.152160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.153792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:12.154435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.155259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.156306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.156678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.156916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.157487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.158821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.161998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.185136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 465/935] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:12.185350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:12.186255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:12.187095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 471/936] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Jun 15 14:36:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:12.191238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.191485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.213799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:12.319058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d18cdf-65e6-4e22-83b3-d8576124f97b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.328223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdea6ac-7147-4455-8c92-a79a1912bc60 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.368709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.369712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.370118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.370558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.372028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:12.372487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.374800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.375183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.376060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.376546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.377210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.382588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 472/937] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:12.391174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.393523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.395437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.409858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:12.431706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:12.432701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:12.433772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:12.433772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:12.433772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:12.433772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:12.439262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 466/938] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:12.440624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.496008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:12.520055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4d71f-e012-4621-9e6a-050d09db377c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.560691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.561523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.561891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.562472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.563116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:12.563521 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.564998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.568900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a5784-0b0e-4814-9efa-67d1ed357c76 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.575004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 473/939] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:12.576426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.609510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.610614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.611114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.611577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.612537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:12.613044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.614778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.634526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 467/940] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:12.635905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.694066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6cfb4-6c19-4cc7-8543-cb216e26aa68 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.704910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:12.741430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260f406-f6d1-4837-be01-5f297b885d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.742096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260f406-f6d1-4837-be01-5f297b885d29 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.742405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260f406-f6d1-4837-be01-5f297b885d29 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.742673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260f406-f6d1-4837-be01-5f297b885d29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.744090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260f406-f6d1-4837-be01-5f297b885d29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.745482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260f406-f6d1-4837-be01-5f297b885d29 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.755511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea683e-1073-492f-8703-ce38d5b44bc2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:12.797287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1b6f-430b-47c9-a158-e6e4306425c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:12.798263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1b6f-430b-47c9-a158-e6e4306425c7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.798667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1b6f-430b-47c9-a158-e6e4306425c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.798955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1b6f-430b-47c9-a158-e6e4306425c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.808677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1b6f-430b-47c9-a158-e6e4306425c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:12.810234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1b6f-430b-47c9-a158-e6e4306425c7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-24fc1b6f-430b-47c9-a158-e6e4306425c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:12.826161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:12.827268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:12.828133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 468/941] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:12.829065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443234c-41cf-48e3-99a0-d76367067988 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.829065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443234c-41cf-48e3-99a0-d76367067988 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.829257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443234c-41cf-48e3-99a0-d76367067988 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.830595 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 469/942] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:12.835093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443234c-41cf-48e3-99a0-d76367067988 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:12.836837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:12.844845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 474/943] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:36:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:12.856469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71565d-0868-4c64-8874-24ec614b605b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:12.856786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71565d-0868-4c64-8874-24ec614b605b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:12.857378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71565d-0868-4c64-8874-24ec614b605b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:12.886918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ec71565d-0868-4c64-8874-24ec614b605b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:12.888760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ec71565d-0868-4c64-8874-24ec614b605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:12.998926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 475/944] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:13.003419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443234c-41cf-48e3-99a0-d76367067988 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:13.021000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71565d-0868-4c64-8874-24ec614b605b None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:13.024975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:13.048780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:13.049514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:13.049814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:13.050140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:13.051462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:13.051462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:13.053721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:13.070994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 470/945] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:13.077622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:13.172828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71565d-0868-4c64-8874-24ec614b605b None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:13.206476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:13.220237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c368cf-c7d4-4ca2-af29-e34490534b6a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:13.225518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:13.225518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:13.225518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:13.225518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:13.226151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:13.226846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:13.229453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING keystone.server.flask.common [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 14:36:13.259103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:13.260141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:13.260504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:13.260788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/fd1b940712c1481ba53ae1b45c68c347/roles/30837f7a6e9442c2a8d667aed7aa1683` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:13.263144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=fd1b940712c1481ba53ae1b45c68c347, role_id=30837f7a6e9442c2a8d667aed7aa1683)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:13.264594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:13.308494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.sql.core [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:36:13.308494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:36:13.308494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [parameters: {'id': '780025dd60e5479b9e876fb9dbfdefe4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:36:13.308494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87000) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 780025dd60e5479b9e876fb9dbfdefe4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 780025dd60e5479b9e876fb9dbfdefe4. Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:36:13.343806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result.read() Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 14:36:13.344719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application yield resource Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise self.value Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application session.commit() Jun 15 14:36:13.345672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:36:13.347253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:36:13.348709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result.read() Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:36:13.350655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [parameters: {'id': '780025dd60e5479b9e876fb9dbfdefe4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:13.352269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:13.354059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 780025dd60e5479b9e876fb9dbfdefe4. Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.355231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 476/946] 10.222.0.23 () {60 vars in 1140 bytes} [Wed Jun 15 14:36:13 2022] POST /identity/v3/projects => generated 262 bytes in 326 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:13.357221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:13.370451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 471/947] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:36:13 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/fd1b940712c1481ba53ae1b45c68c347/roles/30837f7a6e9442c2a8d667aed7aa1683 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:36:13.490669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74cbe8-709a-4ec1-8c80-d8c76561b3c9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:13.538257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166f210-2288-4bad-b1c6-ee1624ff073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:13.538257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166f210-2288-4bad-b1c6-ee1624ff073d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:13.538257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166f210-2288-4bad-b1c6-ee1624ff073d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:13.538257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166f210-2288-4bad-b1c6-ee1624ff073d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:13.542512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166f210-2288-4bad-b1c6-ee1624ff073d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:13.544229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166f210-2288-4bad-b1c6-ee1624ff073d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-6166f210-2288-4bad-b1c6-ee1624ff073d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:13.560947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:13.561934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:13.562858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 477/948] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:13.571559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:13.633095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:13.698683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eced410-9a6e-43a8-b3fd-699595e54909 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:13.735922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7c575-2841-422e-b5bc-0ea44aa2ecb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:13.737047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7c575-2841-422e-b5bc-0ea44aa2ecb7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:13.738047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7c575-2841-422e-b5bc-0ea44aa2ecb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:13.738445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7c575-2841-422e-b5bc-0ea44aa2ecb7 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:13.739385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7c575-2841-422e-b5bc-0ea44aa2ecb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:13.739801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7c575-2841-422e-b5bc-0ea44aa2ecb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:13.741364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7c575-2841-422e-b5bc-0ea44aa2ecb7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:13.761463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 472/949] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:36:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:14.253690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3d512-5ad9-4092-895e-f751ea13c1dd None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:14.254234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3d512-5ad9-4092-895e-f751ea13c1dd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:14.254234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3d512-5ad9-4092-895e-f751ea13c1dd None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:14.258782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 478/950] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:14.264572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a15765-13b9-4910-a138-c1541cb65911 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:14.265188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a15765-13b9-4910-a138-c1541cb65911 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:14.265188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a15765-13b9-4910-a138-c1541cb65911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:14.283878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-24a15765-13b9-4910-a138-c1541cb65911 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:14.285097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-24a15765-13b9-4910-a138-c1541cb65911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:14.369756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 473/951] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:14.393488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3d512-5ad9-4092-895e-f751ea13c1dd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:14.442760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:14.514742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3d512-5ad9-4092-895e-f751ea13c1dd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:14.566199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:14.566199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:14.566199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:14.566199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:14.566659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:14.566659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:14.580350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:14.581512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:14.582442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 479/952] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:14.590862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a15765-13b9-4910-a138-c1541cb65911 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:14.624840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:14.717827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a15765-13b9-4910-a138-c1541cb65911 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:14.754522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:14.755198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:14.755478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:14.755750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:14.756468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:14.756775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:14.758243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:14.764486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 474/953] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:14.770129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:14.884413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e110d-6f19-4cc2-a448-ab7770e4577e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:14.886729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:14.924251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a44c6-42b6-489d-802c-98f9b79a58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:14.924251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a44c6-42b6-489d-802c-98f9b79a58b9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:14.924846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a44c6-42b6-489d-802c-98f9b79a58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:14.924846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a44c6-42b6-489d-802c-98f9b79a58b9 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:14.925304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a44c6-42b6-489d-802c-98f9b79a58b9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:14.925616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a44c6-42b6-489d-802c-98f9b79a58b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:14.927047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a44c6-42b6-489d-802c-98f9b79a58b9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:14.934600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 480/954] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:14.940393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:15.049079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:15.058260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb40fd-b198-4a5e-b90c-0cb763202bde None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:15.096744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b1f96-b0f2-497f-8890-fd01a7877e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:15.097419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b1f96-b0f2-497f-8890-fd01a7877e1d None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.097700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b1f96-b0f2-497f-8890-fd01a7877e1d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.097973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b1f96-b0f2-497f-8890-fd01a7877e1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.098777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b1f96-b0f2-497f-8890-fd01a7877e1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:15.100215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b1f96-b0f2-497f-8890-fd01a7877e1d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:15.184039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 475/955] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:36:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:15.332967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706b6fa-5bdf-46ba-b187-50bb7f14fba1 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.333867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706b6fa-5bdf-46ba-b187-50bb7f14fba1 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.333867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706b6fa-5bdf-46ba-b187-50bb7f14fba1 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.335498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 481/956] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:15.341667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8fd7-db53-467e-af99-b7d98a8df9d2 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.342377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8fd7-db53-467e-af99-b7d98a8df9d2 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.342377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8fd7-db53-467e-af99-b7d98a8df9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.361964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-78fd8fd7-db53-467e-af99-b7d98a8df9d2 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:15.363238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-78fd8fd7-db53-467e-af99-b7d98a8df9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:15.444622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 476/957] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:15.467220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706b6fa-5bdf-46ba-b187-50bb7f14fba1 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:15.584728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706b6fa-5bdf-46ba-b187-50bb7f14fba1 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:15.628513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fc3f2-bcd9-4b14-aca6-c080bc9e695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:15.629340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fc3f2-bcd9-4b14-aca6-c080bc9e695f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.629620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fc3f2-bcd9-4b14-aca6-c080bc9e695f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.629896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fc3f2-bcd9-4b14-aca6-c080bc9e695f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.631990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fc3f2-bcd9-4b14-aca6-c080bc9e695f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:15.632274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fc3f2-bcd9-4b14-aca6-c080bc9e695f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-b48fc3f2-bcd9-4b14-aca6-c080bc9e695f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:15.646795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:15.649631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:15.652243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 482/958] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:15.658194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8fd7-db53-467e-af99-b7d98a8df9d2 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:15.747696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:15.756492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276a75-9524-4442-a2d5-95fec31fb03b None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.756809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276a75-9524-4442-a2d5-95fec31fb03b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.757045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276a75-9524-4442-a2d5-95fec31fb03b None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.759473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 483/959] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:15.770032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68ee41-0bb7-46f0-aeb3-6af0794dc695 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.770032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68ee41-0bb7-46f0-aeb3-6af0794dc695 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.770032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68ee41-0bb7-46f0-aeb3-6af0794dc695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.776928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8fd7-db53-467e-af99-b7d98a8df9d2 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:15.790019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ab68ee41-0bb7-46f0-aeb3-6af0794dc695 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:15.792208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ab68ee41-0bb7-46f0-aeb3-6af0794dc695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:15.823090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:15.824406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:15.825530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:15.825814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:15.826091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:15.827139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] RBAC: Adding query filter params (name=compute) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:15.827581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:15.829158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:15.838297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 477/960] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:36:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:15.841503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:15.888603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 484/961] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:15.911518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68ee41-0bb7-46f0-aeb3-6af0794dc695 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:15.954556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066458ea-229e-4a00-aa96-b565d34d936c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.000135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.001130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.001426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.001799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:16.002897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:16.004287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:16.018502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 478/962] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:36:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:16.049838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68ee41-0bb7-46f0-aeb3-6af0794dc695 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.095069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9ede3-415a-40d4-8702-26696f4ee59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.096635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ede3-415a-40d4-8702-26696f4ee59d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.097169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ede3-415a-40d4-8702-26696f4ee59d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.097710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ede3-415a-40d4-8702-26696f4ee59d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.111228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9ede3-415a-40d4-8702-26696f4ee59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-86f9ede3-415a-40d4-8702-26696f4ee59d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:16.121679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:16.122790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:16.123622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:16.130299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 485/963] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Jun 15 14:36:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:16.133988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:16.162565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:16.263499 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17408-ab5a-486f-8b0c-7a5fe5a07456 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.270799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97ab56-fb1f-479f-b0bb-cff36d55b1c0 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.271087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97ab56-fb1f-479f-b0bb-cff36d55b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.271401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97ab56-fb1f-479f-b0bb-cff36d55b1c0 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.273025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 486/964] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:16.279378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ddd36-a02c-4254-870a-df4f6e4ed7d8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.279661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ddd36-a02c-4254-870a-df4f6e4ed7d8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.279936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ddd36-a02c-4254-870a-df4f6e4ed7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.298861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1f9ddd36-a02c-4254-870a-df4f6e4ed7d8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:16.300044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1f9ddd36-a02c-4254-870a-df4f6e4ed7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:16.319426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.320201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:16.321973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.322477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.323014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.324120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:16.324620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:16.328081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:16.345422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 479/965] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Jun 15 14:36:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:16.348869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:16.395139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 487/966] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:16.422829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ddd36-a02c-4254-870a-df4f6e4ed7d8 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:16.431607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:16.495115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b252ff-b7b3-4c96-aba7-2a6848fef018 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.543434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.544706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.545578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.546056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.559891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:16.560335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ddd36-a02c-4254-870a-df4f6e4ed7d8 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.561894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:16.585976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:16.587675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:16.588577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:16.588577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:16.588577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:16.588577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:16.595804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 480/967] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:16.598324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:16.612741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.613772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.614242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.614829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.615858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:16.617753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:16.668253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.sql.core [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:36:16.668253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:36:16.668253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [parameters: {'user_id': 'ac687e0e2e274324a5af17e306d5ced3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:36:16.668253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87000) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result.read() Jun 15 14:36:16.675795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:36:16.682783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application yield resource Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application next(self.gen) Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise self.value Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application session.commit() Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 14:36:16.683733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.session.flush() Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application compat.raise_( Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 14:36:16.684610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 14:36:16.685485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application conn.query(q) Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application result.read() Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 14:36:16.686401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac687e0e2e274324a5af17e306d5ced3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:16.687227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 14:36:16.688056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 488/968] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Jun 15 14:36:16 2022] POST /identity/v3/users => generated 160 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:16.688943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:16.720348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:16.741659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730fb00-9836-4c12-8f8c-ee468e3dcd47 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.762637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:16.783193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.784253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.784741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.785208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.786407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:16.786946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:16.788553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:16.803553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 481/969] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:16.810555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:16.843796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0d72-1701-4f3e-9eb4-ce135310aa44 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.891502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.893332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.893832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.894550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:16.906638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:16.909775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:16.928857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:16.929866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 14:36:16.931743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:16.938779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 489/970] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:36:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:16.939848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:16.956446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac583c-67ed-4a35-8687-4928777ced7f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:16.996517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:16.997928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:16.998447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:16.998929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.008217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:17.010817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:17.027207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:17.028244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 14:36:17.029254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:17.031499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:17.033069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 482/971] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:36:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:17.039454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:17.066361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d5c17-6b13-404b-9e40-9117b565c137 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:17.082506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:17.108431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:17.109125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.109408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.109766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.110530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:17.110837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:17.112417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:17.159955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7f46f-22b3-43da-ae7b-097c7622f3c4 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:17.170203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 490/972] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Jun 15 14:36:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:17.212710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42025efc-2e77-4bd7-85bc-b222fd45bee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:17.213445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025efc-2e77-4bd7-85bc-b222fd45bee9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.213727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025efc-2e77-4bd7-85bc-b222fd45bee9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.214005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025efc-2e77-4bd7-85bc-b222fd45bee9 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.214975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025efc-2e77-4bd7-85bc-b222fd45bee9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:17.215285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025efc-2e77-4bd7-85bc-b222fd45bee9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:17.216803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025efc-2e77-4bd7-85bc-b222fd45bee9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:17.244625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 483/973] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Jun 15 14:36:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:17.248265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:17.384455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd7742-2b08-46f5-a7d3-ed27df16aaea None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:17.419033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:17.423124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b030ff-7fbd-445a-8d48-6fb2885bcac8 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.423588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b030ff-7fbd-445a-8d48-6fb2885bcac8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.423894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b030ff-7fbd-445a-8d48-6fb2885bcac8 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.425741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 484/974] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:17.426019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:17.426713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.426995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.427330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.433425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3ee48-c39e-4c05-92be-21ed31b0874b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.433784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3ee48-c39e-4c05-92be-21ed31b0874b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.434040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3ee48-c39e-4c05-92be-21ed31b0874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.458490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-0bd3ee48-c39e-4c05-92be-21ed31b0874b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:17.459679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0bd3ee48-c39e-4c05-92be-21ed31b0874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:17.531398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:17.533549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:17.547399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 485/975] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:17.573547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3ee48-c39e-4c05-92be-21ed31b0874b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:17.601609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:17.652215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 491/976] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:17 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:17.658767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:17.702763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:17.702968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3ee48-c39e-4c05-92be-21ed31b0874b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:17.746318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:17.747131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.747408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.747675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.748328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:17.749698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:17.763452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:17.764513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:17.765388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 486/977] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:17.770988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:17.796282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5f985-23fd-4eb2-ba21-8802a2da0695 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:17.841632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:17.842645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.843239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.843783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.859586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:17.861074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:17.885784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:17.886810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:17.887705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:17.887705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:17.887705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:17.887705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:17.887705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 492/978] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:17.890413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:17.892097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:17.903477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047d70-ad63-412a-9fe2-21fb0d9bc28e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:17.950326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:17.951393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:17.951826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:17.952252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:17.953401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:17.953850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:17.956150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:17.963752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 487/979] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:17.969214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:17.995343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:18.017537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2048f1-a069-4d0f-8bd5-dca4a39bb995 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.064132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.065150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.065746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.067038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.067976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:18.068471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.070052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.085348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 493/980] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:18.091993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:18.103023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfa6c4-e525-4091-91e5-5ee9a1590429 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.154162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.154988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.155274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.155553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.156276 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:18.156587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.157970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.163977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 488/981] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:18.170627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:18.198789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:18.226532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85330-e13d-4ee2-976f-ab1dc01e186a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.275123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.275954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.276239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.276529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.284227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.285726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:18.300897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:18.301892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:18.303341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 494/982] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:18.308648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:18.314378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:18.319063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278de75-ccf5-4beb-a2c9-5372773addee None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.359024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08098748-4afc-4cef-aecb-4a2c794a52da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.359681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08098748-4afc-4cef-aecb-4a2c794a52da None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.359964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08098748-4afc-4cef-aecb-4a2c794a52da None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.360234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08098748-4afc-4cef-aecb-4a2c794a52da None admin] PATH_INFO: `/v3/endpoints` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.361044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08098748-4afc-4cef-aecb-4a2c794a52da None admin] RBAC: Adding query filter params (interface=admin, service_id=90487d65674b4282aee85d5ef510abe1, region_id=RegionOne) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:18.361355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08098748-4afc-4cef-aecb-4a2c794a52da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.362766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08098748-4afc-4cef-aecb-4a2c794a52da None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.369126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 489/983] 10.222.0.23 () {56 vars in 1249 bytes} [Wed Jun 15 14:36:18 2022] GET /identity/v3/endpoints?service_id=90487d65674b4282aee85d5ef510abe1&interface=admin®ion_id=RegionOne => generated 193 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:18.370563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587d13-e861-46bc-82d3-2f2185087106 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.370848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587d13-e861-46bc-82d3-2f2185087106 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.371125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa587d13-e861-46bc-82d3-2f2185087106 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.374208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 490/984] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:18.375290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa587d13-e861-46bc-82d3-2f2185087106 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:18.416020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3dbc3-e140-4baf-954d-5c97d84634f0 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.453580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6c38-babf-40d0-8c24-d07ca2111a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.454292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6c38-babf-40d0-8c24-d07ca2111a7d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.454543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6c38-babf-40d0-8c24-d07ca2111a7d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.454821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6c38-babf-40d0-8c24-d07ca2111a7d None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.455606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc6c38-babf-40d0-8c24-d07ca2111a7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:18.455916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc6c38-babf-40d0-8c24-d07ca2111a7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.457357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc6c38-babf-40d0-8c24-d07ca2111a7d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.465746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:18.470163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 495/985] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:18.471516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2404c-9415-418f-a0b8-9c9a1dd19bb8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.471802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2404c-9415-418f-a0b8-9c9a1dd19bb8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.472079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2404c-9415-418f-a0b8-9c9a1dd19bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.489109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-00c2404c-9415-418f-a0b8-9c9a1dd19bb8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:18.490603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-00c2404c-9415-418f-a0b8-9c9a1dd19bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:18.491381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa587d13-e861-46bc-82d3-2f2185087106 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.531421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.532101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.532387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.532667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.533360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.535158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.542605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 491/986] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:36:18 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:36:18.543385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:18.566723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:18.577090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 496/987] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:18.601101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2404c-9415-418f-a0b8-9c9a1dd19bb8 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:18.676249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e401e4b-ea53-4ac8-998d-038f21066e04 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.717104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aebf1-4536-4582-95c9-eedf7d53bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.718103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aebf1-4536-4582-95c9-eedf7d53bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.718601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aebf1-4536-4582-95c9-eedf7d53bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.718887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aebf1-4536-4582-95c9-eedf7d53bfb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.719794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80aebf1-4536-4582-95c9-eedf7d53bfb0 None admin] RBAC: Adding query filter params (role.id=30837f7a6e9442c2a8d667aed7aa1683, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=fd1b940712c1481ba53ae1b45c68c347) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:18.720112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80aebf1-4536-4582-95c9-eedf7d53bfb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.722251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80aebf1-4536-4582-95c9-eedf7d53bfb0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.724197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2404c-9415-418f-a0b8-9c9a1dd19bb8 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:18.730730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 492/988] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:18 2022] GET /identity/v3/role_assignments?user.id=fd1b940712c1481ba53ae1b45c68c347&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=30837f7a6e9442c2a8d667aed7aa1683 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:18.768215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4275f8-f15a-4a0a-a9f1-6bb22b8e6bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:18.768893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4275f8-f15a-4a0a-a9f1-6bb22b8e6bbf None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:18.769174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4275f8-f15a-4a0a-a9f1-6bb22b8e6bbf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:18.769449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4275f8-f15a-4a0a-a9f1-6bb22b8e6bbf None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:18.770279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4275f8-f15a-4a0a-a9f1-6bb22b8e6bbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:18.771752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4275f8-f15a-4a0a-a9f1-6bb22b8e6bbf None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:18.882202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 497/989] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:36:18 2022] POST /identity/v3/services => generated 242 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:36:20.770104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b2d44-52f8-42e7-8991-45c9a955b5f3 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:20.770654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b2d44-52f8-42e7-8991-45c9a955b5f3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:20.770654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b2d44-52f8-42e7-8991-45c9a955b5f3 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:20.772174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 493/990] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:20.778114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a8c3-54ca-4b18-9e47-6f56b52d6cbd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:20.778532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a8c3-54ca-4b18-9e47-6f56b52d6cbd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:20.778726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a8c3-54ca-4b18-9e47-6f56b52d6cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:20.796830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f7f1a8c3-54ca-4b18-9e47-6f56b52d6cbd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:20.797935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f7f1a8c3-54ca-4b18-9e47-6f56b52d6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:20.853130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:20.879285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 498/991] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:20.902336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b2d44-52f8-42e7-8991-45c9a955b5f3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:20.914896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:20.989425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3ece-6187-4ad6-8de6-ff3b15fa6b55 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:20.989710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3ece-6187-4ad6-8de6-ff3b15fa6b55 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:20.990315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3ece-6187-4ad6-8de6-ff3b15fa6b55 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:20.991798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 499/992] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:20.997609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec474885-7462-4931-ac65-aa4666e6bb52 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:20.997931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec474885-7462-4931-ac65-aa4666e6bb52 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:20.998203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec474885-7462-4931-ac65-aa4666e6bb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.016985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ec474885-7462-4931-ac65-aa4666e6bb52 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:21.018230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ec474885-7462-4931-ac65-aa4666e6bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:21.020162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b2d44-52f8-42e7-8991-45c9a955b5f3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.061027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70778830-96fd-40f9-9663-21398298b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.062374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70778830-96fd-40f9-9663-21398298b284 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.062878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70778830-96fd-40f9-9663-21398298b284 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.063398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70778830-96fd-40f9-9663-21398298b284 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.064560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70778830-96fd-40f9-9663-21398298b284 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.065973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70778830-96fd-40f9-9663-21398298b284 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-70778830-96fd-40f9-9663-21398298b284 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:21.080497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:21.081701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:21.082961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 494/993] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Jun 15 14:36:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:21.084209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d2ec-3127-4220-ad43-0da63cbf912b None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.084209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d2ec-3127-4220-ad43-0da63cbf912b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.084209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d2ec-3127-4220-ad43-0da63cbf912b None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.086215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 495/994] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:21.088531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d2ec-3127-4220-ad43-0da63cbf912b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.106568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 500/995] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:21.107869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49e68e-2652-47ef-ab11-91abb3a9b4f0 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.108160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49e68e-2652-47ef-ab11-91abb3a9b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.108439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49e68e-2652-47ef-ab11-91abb3a9b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.127613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6a49e68e-2652-47ef-ab11-91abb3a9b4f0 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:21.128634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6a49e68e-2652-47ef-ab11-91abb3a9b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:21.158905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:21.188356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d2ec-3127-4220-ad43-0da63cbf912b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.195254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 501/996] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:21.196637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49e68e-2652-47ef-ab11-91abb3a9b4f0 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.226717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.227518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.227776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.228059 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.228775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:21.229084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.230491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.235344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 496/997] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:21.236129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.242135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:21.300547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49e68e-2652-47ef-ab11-91abb3a9b4f0 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.336997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.337599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.337875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.338143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] PATH_INFO: `/v3/services/image` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.338828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.338967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7d538-093d-49b2-b1e2-10581b7eb236 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.340255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:21.351650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:21.352669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:21.353480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 502/998] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:21.354383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.375188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.375993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.376278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.376548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.377211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.378639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.381594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:21.397802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:21.402016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:21.402900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 497/999] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:21.404095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.477687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9c0dc-c10f-40c4-ad51-dda32f41e033 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.480093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:21.513161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.514065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.514399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.514676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.515382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:21.515686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.517054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.526187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 503/1000] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:21.527034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.529038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd021a7d-eb74-4fed-bfaa-efef44406361 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.566419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.567067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.567347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.567616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.568325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] RBAC: Adding query filter params (name=image) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:21.568629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.569988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.575923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 498/1001] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:21.577062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.653385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33e1e5-e5ae-45ac-8593-7cbabcb3b3a1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.663919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:21.690451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.691125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.691409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.691686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.692397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:21.692759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.694107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.700013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 504/1002] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:21.700830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.716282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d18bd2-aef8-48c9-9a52-5d22a0e9eb3b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.734452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:21.752710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.753377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.753659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.753947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.754715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.756262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.812046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f2010-f641-4b2a-9ba1-182a1e734c23 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:21.848653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:21.849318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:21.849601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:21.849876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:21.850615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] RBAC: Adding query filter params (type=image) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:21.850925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:21.852871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:21.857502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 505/1003] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:21.858455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:21.862206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 499/1004] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:36:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:21.994693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:22.002572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4fc74-5b77-447b-8cf1-3049302e9e15 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:22.052316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:22.053255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:22.053255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:22.053558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:22.054950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:22.054950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:22.056253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:22.062411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 506/1005] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:36:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:22.068018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:22.190235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac3bc4-de15-4c32-8fc3-7d7b55876239 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:22.216171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:22.228084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaf823-7716-4661-97b4-c8a9c8e2297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:22.228768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaf823-7716-4661-97b4-c8a9c8e2297e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:22.229140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaf823-7716-4661-97b4-c8a9c8e2297e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:22.229329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaf823-7716-4661-97b4-c8a9c8e2297e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:22.230138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaf823-7716-4661-97b4-c8a9c8e2297e None admin] RBAC: Adding query filter params (interface=public, service_id=1f24b2545704454ebc645ce3d2cccbb6, region_id=RegionOne) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:22.230541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaf823-7716-4661-97b4-c8a9c8e2297e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:22.231956 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaf823-7716-4661-97b4-c8a9c8e2297e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:22.242909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 500/1006] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:36:22 2022] GET /identity/v3/endpoints?service_id=1f24b2545704454ebc645ce3d2cccbb6&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:22.248798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:22.366892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b86ac6-1feb-43d9-abc3-6669cbccf210 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:22.408461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80797c98-83a5-40c7-838e-3ae526c3e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:22.409142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80797c98-83a5-40c7-838e-3ae526c3e255 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:22.409423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80797c98-83a5-40c7-838e-3ae526c3e255 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:22.409699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80797c98-83a5-40c7-838e-3ae526c3e255 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:22.410415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80797c98-83a5-40c7-838e-3ae526c3e255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:22.411799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80797c98-83a5-40c7-838e-3ae526c3e255 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:22.420785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 507/1007] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:36:22 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:36:24.059095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbdc5f-e03a-437b-9e99-8548bc4fed16 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.062126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbdc5f-e03a-437b-9e99-8548bc4fed16 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.062126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbdc5f-e03a-437b-9e99-8548bc4fed16 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.064486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 501/1008] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:24.070237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024c20a-eaa5-4074-81a4-57f5691a5e6e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.071508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024c20a-eaa5-4074-81a4-57f5691a5e6e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.071508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024c20a-eaa5-4074-81a4-57f5691a5e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.088721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-4024c20a-eaa5-4074-81a4-57f5691a5e6e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:24.089890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4024c20a-eaa5-4074-81a4-57f5691a5e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:24.105118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:24.159252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 508/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:24.185070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbdc5f-e03a-437b-9e99-8548bc4fed16 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:24.251825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8de63-c42d-4ae3-b40f-0e311c2e6fac None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.252163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8de63-c42d-4ae3-b40f-0e311c2e6fac None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.252389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8de63-c42d-4ae3-b40f-0e311c2e6fac None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.254041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 509/1010] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:24.259461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6ef5a-602c-46d5-9c9f-3a5edba62f39 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.259799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6ef5a-602c-46d5-9c9f-3a5edba62f39 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.260077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6ef5a-602c-46d5-9c9f-3a5edba62f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.277682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-09e6ef5a-602c-46d5-9c9f-3a5edba62f39 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:24.278878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-09e6ef5a-602c-46d5-9c9f-3a5edba62f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:24.299408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbdc5f-e03a-437b-9e99-8548bc4fed16 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:24.338432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:24.339148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.339428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.339701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.340866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:24.342360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:24.344114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 502/1011] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:24.349529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:24.354014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 510/1012] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:24.377384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6ef5a-602c-46d5-9c9f-3a5edba62f39 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:24.408101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:24.455587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1067e02-79f0-4f77-837a-11f85e4d2b54 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:24.466413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:24.486762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6ef5a-602c-46d5-9c9f-3a5edba62f39 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:24.489563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8a83c-35fd-4bd3-a4cf-51b7f0e5598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:24.490251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a83c-35fd-4bd3-a4cf-51b7f0e5598f None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.490536 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a83c-35fd-4bd3-a4cf-51b7f0e5598f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.490811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a83c-35fd-4bd3-a4cf-51b7f0e5598f None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.491515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8a83c-35fd-4bd3-a4cf-51b7f0e5598f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:24.493694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8a83c-35fd-4bd3-a4cf-51b7f0e5598f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:24.524997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e543451-8a45-4b33-8bcc-1228cf46d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:24.525683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e543451-8a45-4b33-8bcc-1228cf46d2ee None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.525970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e543451-8a45-4b33-8bcc-1228cf46d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.526277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e543451-8a45-4b33-8bcc-1228cf46d2ee None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.527010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e543451-8a45-4b33-8bcc-1228cf46d2ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:24.528443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e543451-8a45-4b33-8bcc-1228cf46d2ee None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:24.572301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 503/1013] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:36:24 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:36:24.598760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 511/1014] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:36:24 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:36:24.635042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d9e7c-5d6d-48b5-83dd-298c1649e053 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.635324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d9e7c-5d6d-48b5-83dd-298c1649e053 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.635599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d9e7c-5d6d-48b5-83dd-298c1649e053 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.637308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 504/1015] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:24.643111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db5860-a3b0-4cf4-8f5b-4b6e4e9b8846 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.643418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db5860-a3b0-4cf4-8f5b-4b6e4e9b8846 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.643670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db5860-a3b0-4cf4-8f5b-4b6e4e9b8846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.664133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c6db5860-a3b0-4cf4-8f5b-4b6e4e9b8846 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:24.666126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c6db5860-a3b0-4cf4-8f5b-4b6e4e9b8846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:24.753367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 512/1016] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:24 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:24.778510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d9e7c-5d6d-48b5-83dd-298c1649e053 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:24.891203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d9e7c-5d6d-48b5-83dd-298c1649e053 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:24.903857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:24.929126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:24.930599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:24.930887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:24.931163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:24.931877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:24.933271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:24.945379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:24.947753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:24.948666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 505/1017] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:24.953270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db5860-a3b0-4cf4-8f5b-4b6e4e9b8846 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:25.020984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:25.061004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db5860-a3b0-4cf4-8f5b-4b6e4e9b8846 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:25.098691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:25.099424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:25.099706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:25.099987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:25.100696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] RBAC: Adding query filter params (name=compute) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:25.101000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:25.103271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:25.114193 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 513/1018] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:36:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:25.115488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:25.251213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20adca-2c20-45d0-817f-457d5ade3c50 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:25.288140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c1f09-ea60-4285-b10a-25eda09ea812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:25.288830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c1f09-ea60-4285-b10a-25eda09ea812 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:25.289666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c1f09-ea60-4285-b10a-25eda09ea812 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:25.290650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c1f09-ea60-4285-b10a-25eda09ea812 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:25.291384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c1f09-ea60-4285-b10a-25eda09ea812 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:25.291692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c1f09-ea60-4285-b10a-25eda09ea812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:25.293074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c1f09-ea60-4285-b10a-25eda09ea812 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:25.302436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 506/1019] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Jun 15 14:36:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:25.309237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:25.431042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bf23-c231-42b1-b5b0-5a5181e8120b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:25.463276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:25.469961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d14a0-2141-4603-b50e-448b78128533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:25.470660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d14a0-2141-4603-b50e-448b78128533 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:25.470940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d14a0-2141-4603-b50e-448b78128533 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:25.471212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d14a0-2141-4603-b50e-448b78128533 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:25.471975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d14a0-2141-4603-b50e-448b78128533 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:25.473366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d14a0-2141-4603-b50e-448b78128533 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:25.570885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 514/1020] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:36:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:26.843868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5991b-7a82-4ede-b2f3-4633b7dc1b5e None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.844378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5991b-7a82-4ede-b2f3-4633b7dc1b5e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.844378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5991b-7a82-4ede-b2f3-4633b7dc1b5e None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.845887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 507/1021] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:26.851122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8aa279-fb7c-4fe4-9d59-07fbd40fa3e3 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.851608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8aa279-fb7c-4fe4-9d59-07fbd40fa3e3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.851696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8aa279-fb7c-4fe4-9d59-07fbd40fa3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.868730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc57f-94e7-4797-aace-448ede220798 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.869099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc57f-94e7-4797-aace-448ede220798 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.869296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc57f-94e7-4797-aace-448ede220798 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.869418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7d8aa279-fb7c-4fe4-9d59-07fbd40fa3e3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:26.870691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7d8aa279-fb7c-4fe4-9d59-07fbd40fa3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:26.871086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 508/1022] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:26.876320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a06496-6e38-415f-a99a-dc944c0287c5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:26.876612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a06496-6e38-415f-a99a-dc944c0287c5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:26.876889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a06496-6e38-415f-a99a-dc944c0287c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:26.894392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b9a06496-6e38-415f-a99a-dc944c0287c5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:26.895591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b9a06496-6e38-415f-a99a-dc944c0287c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:26.928179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:26.954389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 515/1023] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:26.969055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 509/1024] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:26.977191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8aa279-fb7c-4fe4-9d59-07fbd40fa3e3 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:26.997024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a06496-6e38-415f-a99a-dc944c0287c5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:27.087693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8aa279-fb7c-4fe4-9d59-07fbd40fa3e3 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.102771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a06496-6e38-415f-a99a-dc944c0287c5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.103040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:27.124593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.125623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.126063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.126522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] PATH_INFO: `/v3/services/image` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.127332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.128808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.138717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.139517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.139798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.140070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.141262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.142740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:27.142847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:27.143813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:27.144896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 516/1025] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Jun 15 14:36:26 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:27.150026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 510/1026] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:27.150580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:27.156075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:27.196348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:27.262045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ceadd-d2db-46e6-8593-34f1d436af62 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.270854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473177ed-f846-45e1-9bf2-c18117548a10 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.302816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.304592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.305879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.306432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.307666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] RBAC: Adding query filter params (name=image) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:27.308157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.310294 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.311240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.312073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.312357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.312634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.313345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.314819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.316018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 517/1027] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:27.321903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:27.367235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:27.424868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b1f94-cda3-4278-9c16-3bec8ef9bbba None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.460324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.463018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.463018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.463018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.463018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] RBAC: Adding query filter params (type=image) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:27.463400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.464681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.473377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 518/1028] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:27.477978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:27.480157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 511/1029] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:36:27 2022] POST /identity/v3/users => generated 285 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:27.608370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6d87e-7363-484d-a518-75b6950dc994 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.612072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:27.663838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188f523-5e64-43fa-8515-5a7bc0393963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.665057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f523-5e64-43fa-8515-5a7bc0393963 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.665343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f523-5e64-43fa-8515-5a7bc0393963 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.665685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188f523-5e64-43fa-8515-5a7bc0393963 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.666439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f523-5e64-43fa-8515-5a7bc0393963 None admin] RBAC: Adding query filter params (interface=public, service_id=e4ed4cc83ceb4afd9564c898fd2aff66, region_id=RegionOne) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:27.666756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f523-5e64-43fa-8515-5a7bc0393963 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.668169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188f523-5e64-43fa-8515-5a7bc0393963 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.674022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 519/1030] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:36:27 2022] GET /identity/v3/endpoints?service_id=e4ed4cc83ceb4afd9564c898fd2aff66&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:27.681358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:27.787990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:27.799497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8ef34-ef47-4c07-a70b-5d04d6448b1b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:27.840427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2fc4c-7780-459d-936c-289b96187503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:27.841104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2fc4c-7780-459d-936c-289b96187503 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.841386 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2fc4c-7780-459d-936c-289b96187503 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.841662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2fc4c-7780-459d-936c-289b96187503 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.842728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2fc4c-7780-459d-936c-289b96187503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:27.844138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2fc4c-7780-459d-936c-289b96187503 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:27.851341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 512/1031] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:36:27 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:36:27.858965 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f962ca-2fd8-4071-9113-7eac2204b032 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.859536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f962ca-2fd8-4071-9113-7eac2204b032 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.859536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f962ca-2fd8-4071-9113-7eac2204b032 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.860939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 520/1032] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:27.867044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4a3b-d7bc-49bc-92c3-5ef4f69f2a9a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:27.868055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4a3b-d7bc-49bc-92c3-5ef4f69f2a9a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:27.868532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4a3b-d7bc-49bc-92c3-5ef4f69f2a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:27.892535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-347b4a3b-d7bc-49bc-92c3-5ef4f69f2a9a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:27.893755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-347b4a3b-d7bc-49bc-92c3-5ef4f69f2a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:27.975522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 513/1033] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:27.999946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f962ca-2fd8-4071-9113-7eac2204b032 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.116847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f962ca-2fd8-4071-9113-7eac2204b032 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:28.152526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:28.153342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.153624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.153899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.165447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:28.173702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:28.174648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:28.175485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:28.180203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 521/1034] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Jun 15 14:36:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:28.181605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4a3b-d7bc-49bc-92c3-5ef4f69f2a9a None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.302229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:28.318794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4a3b-d7bc-49bc-92c3-5ef4f69f2a9a None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:28.362204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:28.362995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.363345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.363692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.364516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:28.364892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:28.366369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:28.380135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 514/1035] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Jun 15 14:36:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:28.382213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.415372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:28.496841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b7117-a510-4f95-b7c6-69b4fc74f5ce None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:28.539672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:28.540485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.540766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.541039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.557326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:28.559200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:28.578624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:28.579594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:28.580472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:28.580472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:28.580472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:28.580472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:28.586204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.586729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 522/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:28.703814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3886cc5-5d96-4bad-89ab-92dd272fe714 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:28.741808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:28.743029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.743310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.743581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.744310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:28.744618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:28.745988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:28.762919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 515/1037] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:28.764628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.850232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:28.870196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b275ee-7692-4022-9a0d-e390f12aabfb None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:28.905097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:28.905902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:28.906204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:28.906491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:28.915234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:28.916693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:28.930975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:28.931903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:36:28.932847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:28.934481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 523/1038] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:36:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:28.937964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:28.948010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:29.044481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c779fa6-e3ce-4a6d-82d1-bd4c572fb4bf None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:29.081516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:29.082627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.083359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.083803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.085214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:29.085618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:29.087805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:29.111254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 516/1039] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:36:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:29.112355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:29.238073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096051-b58c-4bca-b205-37db0c1287d6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:29.273762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:29.274449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.274734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.275034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.286304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:29.344604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:29.346083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:29.449888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 524/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:29.455423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:29.533623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:29.574625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41b32d-789d-4812-af10-f6b96b13c29a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:29.610353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1d873-e700-42c5-af59-ce7838e55efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:29.611159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1d873-e700-42c5-af59-ce7838e55efd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.611445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1d873-e700-42c5-af59-ce7838e55efd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.611722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1d873-e700-42c5-af59-ce7838e55efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.622934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1d873-e700-42c5-af59-ce7838e55efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:29.624435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1d873-e700-42c5-af59-ce7838e55efd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-dbf1d873-e700-42c5-af59-ce7838e55efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:29.643160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:29.649547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:29.650657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:29.650657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:29.650657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:29.650657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:29.650657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 517/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:29.651440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:29.699178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:29.740345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd4e2b-5cf4-4f91-87d1-cd7f26648e1c None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.740553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd4e2b-5cf4-4f91-87d1-cd7f26648e1c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.740804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd4e2b-5cf4-4f91-87d1-cd7f26648e1c None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.746206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 518/1042] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:29.747984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bb492-e7dc-4dca-bf1b-97850865b6cd None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.748270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bb492-e7dc-4dca-bf1b-97850865b6cd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.748548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bb492-e7dc-4dca-bf1b-97850865b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.762493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3de625-9b8b-46de-a830-2ace8cdd31b3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:29.767167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b05bb492-e7dc-4dca-bf1b-97850865b6cd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:29.768352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b05bb492-e7dc-4dca-bf1b-97850865b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:29.798298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:29.799107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.799390 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.799667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.800393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:29.800760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:29.802129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:29.814199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 525/1043] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:29.819456 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:29.850205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 519/1044] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:29.870481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bb492-e7dc-4dca-bf1b-97850865b6cd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:29.922359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:29.930820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:29.934792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df88d34-62f5-485c-860c-91f267637af5 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:29.978308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:29.979105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:29.979391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:29.979670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:29.986277 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bb492-e7dc-4dca-bf1b-97850865b6cd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:29.988275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:29.989792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:30.005508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:30.006515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:30.007412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 526/1045] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:30.013642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.033606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.034465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.034752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.035029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.053472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:30.061671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:30.062802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:30.063665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 520/1046] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:30.070193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.139425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4648bf3-7075-4d6e-86ca-92c56e0bc8b3 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.176060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313808b3-2d88-4009-9b46-26de98ffbafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.176732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313808b3-2d88-4009-9b46-26de98ffbafb None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.177012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313808b3-2d88-4009-9b46-26de98ffbafb None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.177289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313808b3-2d88-4009-9b46-26de98ffbafb None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.178080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313808b3-2d88-4009-9b46-26de98ffbafb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:30.178419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313808b3-2d88-4009-9b46-26de98ffbafb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.179523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eff26-54e7-46e6-ac78-cc0bf437d580 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.180001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313808b3-2d88-4009-9b46-26de98ffbafb None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.187288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:30.191728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 527/1047] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:30.193030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20c2ca-9e81-4bfd-9a6d-eec0acb3de9e None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.193316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20c2ca-9e81-4bfd-9a6d-eec0acb3de9e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.193593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20c2ca-9e81-4bfd-9a6d-eec0acb3de9e None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.194845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 528/1048] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:30.197559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20c2ca-9e81-4bfd-9a6d-eec0acb3de9e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.221020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:30.231173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02d901-70b4-4e58-8bcb-89f4630ad796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.231850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d901-70b4-4e58-8bcb-89f4630ad796 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.232184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d901-70b4-4e58-8bcb-89f4630ad796 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.232475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d901-70b4-4e58-8bcb-89f4630ad796 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.233493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d901-70b4-4e58-8bcb-89f4630ad796 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:30.233811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d901-70b4-4e58-8bcb-89f4630ad796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.235237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d901-70b4-4e58-8bcb-89f4630ad796 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.244414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 521/1049] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:30.245720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe05ea2-cec3-46ec-a2bb-6cab3324e48c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.246007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe05ea2-cec3-46ec-a2bb-6cab3324e48c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.246314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe05ea2-cec3-46ec-a2bb-6cab3324e48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.264196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3fe05ea2-cec3-46ec-a2bb-6cab3324e48c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:30.265305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3fe05ea2-cec3-46ec-a2bb-6cab3324e48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:30.311595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20c2ca-9e81-4bfd-9a6d-eec0acb3de9e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.335307 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 522/1050] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:30.337496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe05ea2-cec3-46ec-a2bb-6cab3324e48c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.352929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.353881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.354481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.354761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.355608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] RBAC: Adding query filter params (role.id=30837f7a6e9442c2a8d667aed7aa1683, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=ac6a657099c14f598d5becd5ab72ee84) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:30.355925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.357386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.363488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 529/1051] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/role_assignments?user.id=ac6a657099c14f598d5becd5ab72ee84&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=30837f7a6e9442c2a8d667aed7aa1683 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:30.364297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.466766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:30.469898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:30.470994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe05ea2-cec3-46ec-a2bb-6cab3324e48c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.483267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96ae04-bfab-4f78-8a03-291c81ac27df None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.514883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.515686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.516245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.516245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] PATH_INFO: `/v3/users/db95897ea17a49ae933a6a43b6c2b96a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.524691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.525506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.525794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.526080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] PATH_INFO: `/v3/services/image` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.526787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.528245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.533091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] RBAC: Authorizing `identity:get_user(user_id=db95897ea17a49ae933a6a43b6c2b96a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.534866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.542520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 523/1052] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/users/db95897ea17a49ae933a6a43b6c2b96a => generated 285 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:30.544734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:30.544935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:30.545888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 14:36:30.547299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:30.551521 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 530/1053] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:30.552449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.666825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d5563-bd80-444a-9df7-2a99a7de07dc None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.671494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc084e9-aeac-41e5-9f77-035bf9160470 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.712810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.713483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.713765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.714044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.714801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:30.715112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.716511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.719951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.720789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.721224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.721643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 531/1054] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:30.721831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.727548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.742883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:30.753235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:30.800082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.801459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.839594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d20838-0b24-4599-93ad-1998715195d7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:30.886496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:30.887167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:30.887466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:30.887722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:30.888440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] RBAC: Adding query filter params (type=image) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:30.888744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:30.890116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:30.895568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 532/1055] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:30.901916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.923171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 524/1056] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:36:30.928967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:30.993738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:30.997202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:31.026037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bbb02-30cc-4526-8bec-7ef96ae2e0fd None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:31.041232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec32b8-838d-4c49-b2fb-d3a1923a982d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:31.063254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a132d-e7c5-410b-a6ba-4bb9a0520732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:31.063854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a132d-e7c5-410b-a6ba-4bb9a0520732 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.064141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a132d-e7c5-410b-a6ba-4bb9a0520732 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.064411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a132d-e7c5-410b-a6ba-4bb9a0520732 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.065155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a132d-e7c5-410b-a6ba-4bb9a0520732 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:31.066930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a132d-e7c5-410b-a6ba-4bb9a0520732 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:31.082647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:31.086703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.086703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.086703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.086703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] RBAC: Authorizing `identity:get_project(project_id=6990a7504f1644f799dd30c92326d9ca)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:31.088101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:31.089878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 525/1057] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:36:30 2022] GET /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:31.095886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:31.186202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 533/1058] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:36:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:31.234242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e299d-a981-4d96-be11-f43abbe511e0 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:31.261781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:31.283361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b892f-a9b8-4873-b3b6-62a903cc95ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:31.284039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b892f-a9b8-4873-b3b6-62a903cc95ed None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:31.284322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b892f-a9b8-4873-b3b6-62a903cc95ed None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:31.284615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b892f-a9b8-4873-b3b6-62a903cc95ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:31.285559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b892f-a9b8-4873-b3b6-62a903cc95ed None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=6990a7504f1644f799dd30c92326d9ca, user.id=db95897ea17a49ae933a6a43b6c2b96a) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:31.285874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b892f-a9b8-4873-b3b6-62a903cc95ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:31.287495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b892f-a9b8-4873-b3b6-62a903cc95ed None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:31.302313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 526/1059] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:31 2022] GET /identity/v3/role_assignments?user.id=db95897ea17a49ae933a6a43b6c2b96a&scope.project.id=6990a7504f1644f799dd30c92326d9ca&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:32.511491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2aa5c-d91e-487d-aeb8-b1345632402c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:32.512014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2aa5c-d91e-487d-aeb8-b1345632402c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:32.512125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2aa5c-d91e-487d-aeb8-b1345632402c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:32.513447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 534/1060] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:32.518751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a39e5-65e9-4660-836e-d4ca67625c3e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:32.519166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a39e5-65e9-4660-836e-d4ca67625c3e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:32.519354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a39e5-65e9-4660-836e-d4ca67625c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:32.537209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-041a39e5-65e9-4660-836e-d4ca67625c3e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:32.538449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-041a39e5-65e9-4660-836e-d4ca67625c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:32.616135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 527/1061] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:32.639254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2aa5c-d91e-487d-aeb8-b1345632402c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:32.747881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2aa5c-d91e-487d-aeb8-b1345632402c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:32.778445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:32.783823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:32.784638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:32.784922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:32.785196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:32.800827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:32.809168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:32.810201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:32.811229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 535/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Jun 15 14:36:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:32.818663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a39e5-65e9-4660-836e-d4ca67625c3e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:32.931194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a39e5-65e9-4660-836e-d4ca67625c3e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:32.946747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:32.972349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:32.973089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:32.973368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:32.973736 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:32.974520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:32.974829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:32.976236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:32.990069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 528/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Jun 15 14:36:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:32.991615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.098849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf0125-5ed7-4f31-9f12-16aee18f9352 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:33.133528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:33.134429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.134711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.134984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.146895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:33.148555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:33.166944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:33.167905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:33.168860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:33.168860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:33.168860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:33.168860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:33.168860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 536/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:33.175194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.286707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f27599-fd66-4486-a8dc-e8e1b134b5f2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:33.323050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3c004-9399-41e7-bc54-bc90a0cca718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:33.323770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3c004-9399-41e7-bc54-bc90a0cca718 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.324057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3c004-9399-41e7-bc54-bc90a0cca718 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.324336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3c004-9399-41e7-bc54-bc90a0cca718 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.325218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3c004-9399-41e7-bc54-bc90a0cca718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:33.325529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3c004-9399-41e7-bc54-bc90a0cca718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:33.326953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3c004-9399-41e7-bc54-bc90a0cca718 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:33.340197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 529/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:33.345760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.400066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:33.449470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb703b-9ad1-4014-a163-24b56d0b901e None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.449820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb703b-9ad1-4014-a163-24b56d0b901e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.450009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb703b-9ad1-4014-a163-24b56d0b901e None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.451634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 530/1066] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:33.456559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7ea0e-f3f0-418d-83f7-aa7b0f9e1111 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.456849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7ea0e-f3f0-418d-83f7-aa7b0f9e1111 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.457427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7ea0e-f3f0-418d-83f7-aa7b0f9e1111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.465392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4703-4901-497d-a9b9-bd2a1a1191fe None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:33.475454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-f3b7ea0e-f3f0-418d-83f7-aa7b0f9e1111 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:33.478154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f3b7ea0e-f3f0-418d-83f7-aa7b0f9e1111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:33.501911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e174282-910d-44f8-9d67-30485c13feb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:33.502812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174282-910d-44f8-9d67-30485c13feb4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.503122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174282-910d-44f8-9d67-30485c13feb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.503399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e174282-910d-44f8-9d67-30485c13feb4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.511180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e174282-910d-44f8-9d67-30485c13feb4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:33.511453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:33.512806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e174282-910d-44f8-9d67-30485c13feb4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-4e174282-910d-44f8-9d67-30485c13feb4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:33.527665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:33.528877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 537/1067] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:36:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f2a4-17b7-4066-8b79-40f64f5361bb None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.530642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f2a4-17b7-4066-8b79-40f64f5361bb None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.531890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f2a4-17b7-4066-8b79-40f64f5361bb None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.533495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 538/1068] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:33.536626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289f2a4-17b7-4066-8b79-40f64f5361bb None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.564835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 531/1069] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:33.566271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8011-2cd7-4656-93a7-642994d44afd None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.566571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8011-2cd7-4656-93a7-642994d44afd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.566891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8011-2cd7-4656-93a7-642994d44afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.590486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1ecd8011-2cd7-4656-93a7-642994d44afd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:33.592207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1ecd8011-2cd7-4656-93a7-642994d44afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:33.660012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289f2a4-17b7-4066-8b79-40f64f5361bb None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:33.661717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 532/1070] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:33.662583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd8011-2cd7-4656-93a7-642994d44afd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.675020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:33.696568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:33.697246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.697576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.697855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.698739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:33.699050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:33.700500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:33.720271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 539/1071] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:36:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:33.721096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.786094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:33.793376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd8011-2cd7-4656-93a7-642994d44afd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:33.822319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bd1fc-12f1-444a-80b2-e397ced4f1be None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:33.828485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:33.829143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.829430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.829710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.857685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:33.858504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:33.858784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:33.859052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:33.869966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:33.881062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:33.882157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:33.883723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 540/1072] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:36:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:33.884569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:33.905342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:33.906863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:33.911268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:33.988609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63930920-00ed-4bf0-b754-8447f985e72c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.000606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.007295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 533/1073] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:34.008806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.023395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.024196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.024479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.024752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.035723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.037232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:34.055923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:34.056929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:34.062753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:34.062753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:34.062753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:34.062753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:34.062753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 541/1074] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:34.062753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.110533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6497e-cde1-47d2-8517-a517a4cb5361 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.137376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.144733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.146724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.146724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.146724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.146986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:34.147069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.148539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.156836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 534/1075] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:34.157636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.161928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da4f62-d5de-4786-ad0d-df5b72020274 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.197045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.197840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.198122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.198458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.209457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.210949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.216536 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:34.228091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:34.229220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:34.230136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:34.230136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:34.230136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:34.230136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:34.230136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 542/1076] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:34.230136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.263016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ac0a6-2623-404e-9df1-56d6d5e59f36 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.297954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.298770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.299053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.299391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.300111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:34.300445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.301816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.320169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 535/1077] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:34.320169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.336820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed629-8542-4e10-a5b3-4276e4465f1f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.362318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.373867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.374933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.375438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.375914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] PATH_INFO: `/v3/users/db95897ea17a49ae933a6a43b6c2b96a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.377432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] RBAC: Authorizing `identity:get_user(user_id=db95897ea17a49ae933a6a43b6c2b96a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.381850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.381850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 543/1078] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/users/db95897ea17a49ae933a6a43b6c2b96a => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:34.381850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.426315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ab98c-af0c-46fb-8cc1-c3e2ebe47448 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.443753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.461176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.461838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.462149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.462436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.463155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:34.463458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.464832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.476082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 536/1079] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:34.477535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.484504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58e2e1-74ef-4e99-a9a7-16f49a178a6e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.526251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.530818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.531105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.531369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.542731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.544834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:34.560375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:34.561515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:34.562474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 544/1080] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:34.567775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.626132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee31e-4483-4b26-bc73-6858ebff52c3 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.630709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.669830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.670755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.671250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.671368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.673446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] RBAC: Authorizing `identity:get_project(project_id=6990a7504f1644f799dd30c92326d9ca)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.673961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.675718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 537/1081] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca => generated 311 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:34.681925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.682073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21d54a-6d98-4bcf-8694-695b9edbb522 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.727964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.728643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.728934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.729208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.730009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:34.730354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.732291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.738580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.745684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 545/1082] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:34.752847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:34.802435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f9aaa-6ea6-401d-ad1b-ca77c3f7c21d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.838003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2458872-8a45-4797-9c14-d6b892539f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.838931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2458872-8a45-4797-9c14-d6b892539f21 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.839213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2458872-8a45-4797-9c14-d6b892539f21 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.839490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2458872-8a45-4797-9c14-d6b892539f21 None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca/users/db95897ea17a49ae933a6a43b6c2b96a/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.841487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2458872-8a45-4797-9c14-d6b892539f21 None admin] RBAC: Authorizing `identity:create_grant(project_id=6990a7504f1644f799dd30c92326d9ca, user_id=db95897ea17a49ae933a6a43b6c2b96a, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.842973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2458872-8a45-4797-9c14-d6b892539f21 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.860848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7001-3723-4ec7-9a72-effbe559a700 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:34.874926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:34.899950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552165d-b2bd-4d86-a71d-4fcf88c6e649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:34.900852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552165d-b2bd-4d86-a71d-4fcf88c6e649 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:34.901132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552165d-b2bd-4d86-a71d-4fcf88c6e649 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:34.901404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552165d-b2bd-4d86-a71d-4fcf88c6e649 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/ac6a657099c14f598d5becd5ab72ee84/roles/30837f7a6e9442c2a8d667aed7aa1683` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:34.906374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552165d-b2bd-4d86-a71d-4fcf88c6e649 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=ac6a657099c14f598d5becd5ab72ee84, role_id=30837f7a6e9442c2a8d667aed7aa1683)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:34.908583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552165d-b2bd-4d86-a71d-4fcf88c6e649 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:34.912961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 538/1083] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:36:34 2022] PUT /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca/users/db95897ea17a49ae933a6a43b6c2b96a/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:36:34.999021 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 546/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:36:34 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/ac6a657099c14f598d5becd5ab72ee84/roles/30837f7a6e9442c2a8d667aed7aa1683 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:36:37.203906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82679a-639c-49fc-bfe4-25e38e81decd None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.204448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82679a-639c-49fc-bfe4-25e38e81decd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.204448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82679a-639c-49fc-bfe4-25e38e81decd None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.205829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 539/1085] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:37.212065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bdbf3-2d1d-40e4-bf7b-fc87e273cf61 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.212588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bdbf3-2d1d-40e4-bf7b-fc87e273cf61 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.212588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bdbf3-2d1d-40e4-bf7b-fc87e273cf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.230868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-067bdbf3-2d1d-40e4-bf7b-fc87e273cf61 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:37.232106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-067bdbf3-2d1d-40e4-bf7b-fc87e273cf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:37.307171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 547/1086] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:37.334104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82679a-639c-49fc-bfe4-25e38e81decd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.362556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01cb2-312a-4441-b3d5-fec37230f641 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.362829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01cb2-312a-4441-b3d5-fec37230f641 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.363116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01cb2-312a-4441-b3d5-fec37230f641 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.364499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 548/1087] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:37.372666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7db1-bb9d-407c-a8c7-5e8009755c06 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.372954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7db1-bb9d-407c-a8c7-5e8009755c06 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.373229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7db1-bb9d-407c-a8c7-5e8009755c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.408345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-cbfc7db1-bb9d-407c-a8c7-5e8009755c06 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:37.409564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cbfc7db1-bb9d-407c-a8c7-5e8009755c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:37.481953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 549/1088] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:37.505797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7db1-bb9d-407c-a8c7-5e8009755c06 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.508083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82679a-639c-49fc-bfe4-25e38e81decd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:37.515421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:37.543559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:37.544588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:37.545514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.546009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.546480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.557709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:37.565518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:37.566569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:37.567410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 540/1089] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:37.572887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.611206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7db1-bb9d-407c-a8c7-5e8009755c06 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:37.648481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:37.649538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.649822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.650100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.660797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:37.669700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:37.670673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:37.671696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 550/1090] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:37.676877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.680219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e684d-1ec2-47f0-9f6a-9650f20d2abf None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:37.721874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:37.722577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.722862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.723140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.723900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:37.724208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:37.725676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:37.735105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 541/1091] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:37.740933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.771251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:37.784210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:37.791099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c0ff8-3848-4bf7-8695-c6c51aba704e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:37.827120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:37.827804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.828086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.828363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.829121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:37.829434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:37.830870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:37.841861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 551/1092] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:37.848064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.851368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c0c17-30b7-4028-8f7c-c8b923b3e48d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:37.886503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:37.887405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.887686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.887958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] PATH_INFO: `/v3/users/db95897ea17a49ae933a6a43b6c2b96a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.889256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] RBAC: Authorizing `identity:get_user(user_id=db95897ea17a49ae933a6a43b6c2b96a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:37.890780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:37.892510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 542/1093] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/users/db95897ea17a49ae933a6a43b6c2b96a => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:37.898852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:37.950901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12797a-3c42-44ec-be1c-80e8d2c6b1b7 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:37.985417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:37.986126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:37.986429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:37.986700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:37.993667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:37.996851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:37.998375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:37.999707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3f3b2-b0ad-496a-81bc-806e61b543de None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:38.016548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:38.017497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:38.018592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:38.018592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:38.018592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:38.018592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:38.018592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 552/1094] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:38.024249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:38.034827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:38.035226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:38.036247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:38.036538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:38.036816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:38.104250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:38.105716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:38.124108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e802-e233-41b4-90fc-bf063b31d4f8 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:38.158364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:38.159006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:38.159297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:38.159581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:38.160303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:38.160619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:38.162000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:38.173247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 553/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:38.178891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:38.199027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:38.205800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 543/1096] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:38.212264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:38.245375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:38.278779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451498f8-3872-4eb9-bae3-710e6f90c269 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:38.311899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ae4cf-0ac3-4d6e-a3a6-c3d046d2be7e None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:38.315194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:38.315986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:38.316265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:38.316542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:38.324959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:38.326455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:36:38.341258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:38.342340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:38.343265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 554/1097] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Jun 15 14:36:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:36:38.348065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:38.349095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:38.349962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:38.350270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:38.350553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:38.352445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] RBAC: Authorizing `identity:get_project(project_id=6990a7504f1644f799dd30c92326d9ca)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:38.353822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:38.355567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 544/1098] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:36:38 2022] GET /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:38.361451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:38.425785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:38.455961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2973cad-7ab0-4d62-89fd-89a64a289c9e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:38.464048 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679fd8d-ad12-44a5-a5c3-b219781bffa1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:38.483208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:38.490820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:38.491573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:38.491909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:38.492295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:38.493167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:38.493472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:38.495000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:38.497561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:38.498162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:38.498470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:38.498734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:38.499590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=6990a7504f1644f799dd30c92326d9ca, user.id=db95897ea17a49ae933a6a43b6c2b96a) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:38.499903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:38.501318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:38.508627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 545/1099] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:38 2022] GET /identity/v3/role_assignments?user.id=db95897ea17a49ae933a6a43b6c2b96a&scope.project.id=6990a7504f1644f799dd30c92326d9ca&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:38.517019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 555/1100] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Jun 15 14:36:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:38.519594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:38.646137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa89999-fc76-4106-a5d1-c6b118263151 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:38.675292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:38.684776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:38.686872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:38.686872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:38.686872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:38.759700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:38.759700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:38.857474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 546/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:38.863372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:38.967046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92064ce2-3ba2-47cc-bbec-999791207610 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:39.002064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:39.002895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:39.003182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:39.003459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:39.015038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:39.016464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:36:39.041821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:39.042840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:39.043721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:36:39.043721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:36:39.043721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:36:39.043721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:39.043721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 556/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:39.048908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:39.077509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:39.153357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ec961-e37d-4144-b7f3-7618d80ee2cf None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:39.189190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:39.189852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:39.190131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:39.190432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:39.191161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:39.191464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:39.192840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:39.205043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 547/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:36:39.211917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:39.252974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:39.325787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9491764-9dfa-4155-8f53-a20efcff8f5e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:39.368406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:39.368406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:39.368406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:39.368406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:39.375500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:39.376960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:36:39.391522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:39.392624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:36:39.393835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 557/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:36:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:39.398817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:39.488070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:39.500644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60792ab1-c25a-4492-aad5-3dced35b3f6b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:39.534827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f9c3b-6bf9-44fe-8790-e38d66bd9e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:39.535503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9c3b-6bf9-44fe-8790-e38d66bd9e14 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:39.535944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9c3b-6bf9-44fe-8790-e38d66bd9e14 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:39.536642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9c3b-6bf9-44fe-8790-e38d66bd9e14 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:39.537430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f9c3b-6bf9-44fe-8790-e38d66bd9e14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:39.537741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f9c3b-6bf9-44fe-8790-e38d66bd9e14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:39.540386 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f9c3b-6bf9-44fe-8790-e38d66bd9e14 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:39.553431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 548/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:36:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:39.560459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:39.661394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:39.665661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f481e-5b92-46c9-ad52-a609f6c9ff12 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:39.699132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f851a0-795a-402a-b60b-a4a223f8e199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:39.699840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f851a0-795a-402a-b60b-a4a223f8e199 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:39.700477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f851a0-795a-402a-b60b-a4a223f8e199 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:39.700979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f851a0-795a-402a-b60b-a4a223f8e199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:39.702213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f851a0-795a-402a-b60b-a4a223f8e199 None admin] RBAC: Adding query filter params (role.id=30837f7a6e9442c2a8d667aed7aa1683, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=ac6a657099c14f598d5becd5ab72ee84) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:39.702535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f851a0-795a-402a-b60b-a4a223f8e199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:39.703989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f851a0-795a-402a-b60b-a4a223f8e199 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:39.714341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 558/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:39 2022] GET /identity/v3/role_assignments?user.id=ac6a657099c14f598d5becd5ab72ee84&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=30837f7a6e9442c2a8d667aed7aa1683 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:40.714610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a83d70-acc1-4ad6-a251-83d5d425d922 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:40.715173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a83d70-acc1-4ad6-a251-83d5d425d922 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:40.715173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a83d70-acc1-4ad6-a251-83d5d425d922 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:40.716511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 549/1107] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:40.721679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a086c2-635e-4781-a8c2-dfbab271be90 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:40.722107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a086c2-635e-4781-a8c2-dfbab271be90 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:40.722261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a086c2-635e-4781-a8c2-dfbab271be90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:40.738718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-05a086c2-635e-4781-a8c2-dfbab271be90 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:40.739896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-05a086c2-635e-4781-a8c2-dfbab271be90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:40.804924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 559/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:40.829358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a83d70-acc1-4ad6-a251-83d5d425d922 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:40.928997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a83d70-acc1-4ad6-a251-83d5d425d922 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:40.964045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9045b-92c8-488d-8c2c-1add5279dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:40.964838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9045b-92c8-488d-8c2c-1add5279dbea None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:40.965151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9045b-92c8-488d-8c2c-1add5279dbea None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:40.965429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9045b-92c8-488d-8c2c-1add5279dbea None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:40.966626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9045b-92c8-488d-8c2c-1add5279dbea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:40.968053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9045b-92c8-488d-8c2c-1add5279dbea None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:40.969657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 550/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:40.975459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a086c2-635e-4781-a8c2-dfbab271be90 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:41.063819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:41.072634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a086c2-635e-4781-a8c2-dfbab271be90 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:41.106523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5de10-9b63-4d6f-9115-57771b375661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:41.107186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5de10-9b63-4d6f-9115-57771b375661 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:41.107466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5de10-9b63-4d6f-9115-57771b375661 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:41.107739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5de10-9b63-4d6f-9115-57771b375661 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:41.108446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5de10-9b63-4d6f-9115-57771b375661 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:41.109890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5de10-9b63-4d6f-9115-57771b375661 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:41.209660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 560/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:36:40 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:43.426367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be915a33-4ea8-40eb-ae92-1f2d69d33c08 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:43.427008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be915a33-4ea8-40eb-ae92-1f2d69d33c08 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:43.427008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be915a33-4ea8-40eb-ae92-1f2d69d33c08 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:43.428280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 551/1111] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:43.434297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51ca70-9639-436a-8228-5f662f58e9a5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:43.434745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51ca70-9639-436a-8228-5f662f58e9a5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:43.434885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51ca70-9639-436a-8228-5f662f58e9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:43.452356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6f51ca70-9639-436a-8228-5f662f58e9a5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:43.453552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6f51ca70-9639-436a-8228-5f662f58e9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:43.521278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 561/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:43.544566 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be915a33-4ea8-40eb-ae92-1f2d69d33c08 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:43.572463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:43.661486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be915a33-4ea8-40eb-ae92-1f2d69d33c08 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:43.704020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:43.704884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:43.705171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:43.705448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] PATH_INFO: `/v3/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:43.706979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] RBAC: Authorizing `identity:get_role(role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:43.708625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:43.710044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 552/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:43 2022] GET /identity/v3/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:43.715871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51ca70-9639-436a-8228-5f662f58e9a5 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:43.794250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:43.816135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51ca70-9639-436a-8228-5f662f58e9a5 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:43.854918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:43.855795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:43.856079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:43.856357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] PATH_INFO: `/v3/users/d037ea44dbe74a95b85458f0cebac9a1` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:43.870770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] RBAC: Authorizing `identity:get_user(user_id=d037ea44dbe74a95b85458f0cebac9a1)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:43.872239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:43.873996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 562/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:43 2022] GET /identity/v3/users/d037ea44dbe74a95b85458f0cebac9a1 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:43.882208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:43.968375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:43.980486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9abef7-0d88-4fe1-9718-02ae1bb3fb22 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:44.015161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:44.015805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:44.016087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:44.016369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:44.084196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:44.085883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:44.170107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:44.186898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 553/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:36:44.197748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:44.299691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665ef82-d8e9-4dbe-a8e9-2523e6bb64f6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:44.336084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2067-602c-4d56-a082-2199f7f40a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:44.336944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2067-602c-4d56-a082-2199f7f40a73 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:44.337231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2067-602c-4d56-a082-2199f7f40a73 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:44.337509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2067-602c-4d56-a082-2199f7f40a73 None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:44.338764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2067-602c-4d56-a082-2199f7f40a73 None admin] RBAC: Authorizing `identity:get_project(project_id=6990a7504f1644f799dd30c92326d9ca)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:44.340250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2067-602c-4d56-a082-2199f7f40a73 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:44.341951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 563/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:36:44 2022] GET /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:44.348595 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:44.490780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46065f03-887d-4a00-aaf5-241996d112c7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:44.526616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56607d13-8d1f-4bbd-ace0-e09ab57a0234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:44.527270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56607d13-8d1f-4bbd-ace0-e09ab57a0234 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:44.527550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56607d13-8d1f-4bbd-ace0-e09ab57a0234 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:44.527827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56607d13-8d1f-4bbd-ace0-e09ab57a0234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:44.528651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56607d13-8d1f-4bbd-ace0-e09ab57a0234 None admin] RBAC: Adding query filter params (role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=6990a7504f1644f799dd30c92326d9ca, user.id=d037ea44dbe74a95b85458f0cebac9a1) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:44.528965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56607d13-8d1f-4bbd-ace0-e09ab57a0234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:44.530411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56607d13-8d1f-4bbd-ace0-e09ab57a0234 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:44.536046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 554/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:44 2022] GET /identity/v3/role_assignments?user.id=d037ea44dbe74a95b85458f0cebac9a1&scope.project.id=6990a7504f1644f799dd30c92326d9ca&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:46.692211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce5a7b-4bf9-468f-96f7-fdb93df7aa7b None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:46.692791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce5a7b-4bf9-468f-96f7-fdb93df7aa7b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:46.692791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce5a7b-4bf9-468f-96f7-fdb93df7aa7b None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:46.694153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 564/1118] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:36:46.701418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083058f5-9824-4d18-bf62-5f58e50c6508 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:46.701903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083058f5-9824-4d18-bf62-5f58e50c6508 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:46.702013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083058f5-9824-4d18-bf62-5f58e50c6508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:46.719054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-083058f5-9824-4d18-bf62-5f58e50c6508 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:46.720257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-083058f5-9824-4d18-bf62-5f58e50c6508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:46.764783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:46.787598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 555/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:46.810985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce5a7b-4bf9-468f-96f7-fdb93df7aa7b None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:46.814813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:46.910006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce5a7b-4bf9-468f-96f7-fdb93df7aa7b None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:46.943949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:46.944911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:46.945187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:46.945459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] PATH_INFO: `/v3/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:46.946952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] RBAC: Authorizing `identity:get_role(role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:46.948329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:46.949703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 565/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:46 2022] GET /identity/v3/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:46.955982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083058f5-9824-4d18-bf62-5f58e50c6508 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:47.059062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083058f5-9824-4d18-bf62-5f58e50c6508 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:47.094559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:47.095426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:47.095705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:47.095977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] PATH_INFO: `/v3/users/d037ea44dbe74a95b85458f0cebac9a1` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:47.097159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] RBAC: Authorizing `identity:get_user(user_id=d037ea44dbe74a95b85458f0cebac9a1)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:47.098629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:47.100260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 556/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:46 2022] GET /identity/v3/users/d037ea44dbe74a95b85458f0cebac9a1 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:47.107221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:47.178793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:47.209541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e45b0-8f84-45a7-b406-9a3b04d1620b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:47.244433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1491087-b945-4483-872a-e6803daf3ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:47.245281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1491087-b945-4483-872a-e6803daf3ef4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:47.245557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1491087-b945-4483-872a-e6803daf3ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:47.245826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1491087-b945-4483-872a-e6803daf3ef4 None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:47.247076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1491087-b945-4483-872a-e6803daf3ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=6990a7504f1644f799dd30c92326d9ca)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:47.248526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1491087-b945-4483-872a-e6803daf3ef4 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:47.250264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 566/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:36:47 2022] GET /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:47.256203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:47.304271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:47.357454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e2478-ee65-458e-9cd6-f82c6816e01d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:47.389760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0078-10f7-4a50-b24a-6b16e0d66667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:47.390678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0078-10f7-4a50-b24a-6b16e0d66667 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:47.390961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0078-10f7-4a50-b24a-6b16e0d66667 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:47.391237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0078-10f7-4a50-b24a-6b16e0d66667 None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca/users/d037ea44dbe74a95b85458f0cebac9a1/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:47.393405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada0078-10f7-4a50-b24a-6b16e0d66667 None admin] RBAC: Authorizing `identity:create_grant(project_id=6990a7504f1644f799dd30c92326d9ca, user_id=d037ea44dbe74a95b85458f0cebac9a1, role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:47.394877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada0078-10f7-4a50-b24a-6b16e0d66667 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:47.507321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 557/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:36:47 2022] PUT /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca/users/d037ea44dbe74a95b85458f0cebac9a1/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:36:49.668645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39bca9-a039-418b-8038-df4c936bbcdb None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:49.669208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39bca9-a039-418b-8038-df4c936bbcdb None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:49.669208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39bca9-a039-418b-8038-df4c936bbcdb None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:49.670609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 567/1124] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:49.675985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17efc3-a748-40c1-a58f-423faccb4a34 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:49.676439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17efc3-a748-40c1-a58f-423faccb4a34 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:49.676529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17efc3-a748-40c1-a58f-423faccb4a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:49.694129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4d17efc3-a748-40c1-a58f-423faccb4a34 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:49.696417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4d17efc3-a748-40c1-a58f-423faccb4a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:49.766691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 558/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:49.790998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39bca9-a039-418b-8038-df4c936bbcdb None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:49.891928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39bca9-a039-418b-8038-df4c936bbcdb None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:49.926516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:49.927365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:49.927643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:49.927915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] PATH_INFO: `/v3/roles/1f021ccb57a048c4b2d194fdba7e38f0` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:49.930432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] RBAC: Authorizing `identity:get_role(role_id=1f021ccb57a048c4b2d194fdba7e38f0)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:49.931814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:49.933214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 568/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:49 2022] GET /identity/v3/roles/1f021ccb57a048c4b2d194fdba7e38f0 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:49.939059 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17efc3-a748-40c1-a58f-423faccb4a34 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:49.978855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:50.045111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17efc3-a748-40c1-a58f-423faccb4a34 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:50.079944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:50.080806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:50.081087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:50.081358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] PATH_INFO: `/v3/users/d037ea44dbe74a95b85458f0cebac9a1` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:50.082544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=d037ea44dbe74a95b85458f0cebac9a1)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:50.084321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:50.086070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 559/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:49 2022] GET /identity/v3/users/d037ea44dbe74a95b85458f0cebac9a1 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:50.091833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:50.103391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:50.190408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a7228-8a34-4036-be88-638e1c8e1dda None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:50.224510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:50.225195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:50.226122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:50.226514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:50.295464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:50.296855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:50.307009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:50.395812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 569/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:50.402920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:50.509296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824df842-ee43-44d7-bcd6-e73ffd46f0f2 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:50.511702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:50.543954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a31b7c-fcfd-4abf-8747-8fe00aca6802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:50.544812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a31b7c-fcfd-4abf-8747-8fe00aca6802 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:50.545110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a31b7c-fcfd-4abf-8747-8fe00aca6802 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:50.545647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a31b7c-fcfd-4abf-8747-8fe00aca6802 None admin] PATH_INFO: `/v3/projects/6990a7504f1644f799dd30c92326d9ca` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:50.547320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a31b7c-fcfd-4abf-8747-8fe00aca6802 None admin] RBAC: Authorizing `identity:get_project(project_id=6990a7504f1644f799dd30c92326d9ca)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:50.548774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a31b7c-fcfd-4abf-8747-8fe00aca6802 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:50.550540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 560/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:36:50 2022] GET /identity/v3/projects/6990a7504f1644f799dd30c92326d9ca => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:50.557385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:50.658065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7aeaa1-d87a-4835-b662-4cff358a350e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:50.667839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:50.692835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf27c9-9daf-4d8b-8ab6-2ef1ae3997a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:50.693511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf27c9-9daf-4d8b-8ab6-2ef1ae3997a0 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:50.693760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf27c9-9daf-4d8b-8ab6-2ef1ae3997a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:50.694786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf27c9-9daf-4d8b-8ab6-2ef1ae3997a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:50.695639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf27c9-9daf-4d8b-8ab6-2ef1ae3997a0 None admin] RBAC: Adding query filter params (role.id=1f021ccb57a048c4b2d194fdba7e38f0, scope.project.id=6990a7504f1644f799dd30c92326d9ca, user.id=d037ea44dbe74a95b85458f0cebac9a1) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:50.695949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf27c9-9daf-4d8b-8ab6-2ef1ae3997a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:50.697396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf27c9-9daf-4d8b-8ab6-2ef1ae3997a0 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:50.704107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 570/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:50 2022] GET /identity/v3/role_assignments?user.id=d037ea44dbe74a95b85458f0cebac9a1&scope.project.id=6990a7504f1644f799dd30c92326d9ca&role.id=1f021ccb57a048c4b2d194fdba7e38f0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:52.869395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40005e8e-d51c-4bfe-8e3f-32ac5f5579eb None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:52.869918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40005e8e-d51c-4bfe-8e3f-32ac5f5579eb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:52.869918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40005e8e-d51c-4bfe-8e3f-32ac5f5579eb None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:52.871342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 561/1131] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:52.876797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c6d74-184e-4b8e-807f-153ffc980f97 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:52.877242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c6d74-184e-4b8e-807f-153ffc980f97 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:52.877344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c6d74-184e-4b8e-807f-153ffc980f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:52.894127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-2c3c6d74-184e-4b8e-807f-153ffc980f97 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:52.895345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2c3c6d74-184e-4b8e-807f-153ffc980f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:52.963073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 571/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:52.987492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40005e8e-d51c-4bfe-8e3f-32ac5f5579eb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:53.092238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40005e8e-d51c-4bfe-8e3f-32ac5f5579eb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:53.127267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229b4f-6e20-4bde-9cf1-64540b9ccda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:53.127966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229b4f-6e20-4bde-9cf1-64540b9ccda5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:53.128246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229b4f-6e20-4bde-9cf1-64540b9ccda5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:53.128519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229b4f-6e20-4bde-9cf1-64540b9ccda5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:53.129674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229b4f-6e20-4bde-9cf1-64540b9ccda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:53.131270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229b4f-6e20-4bde-9cf1-64540b9ccda5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:53.132926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 562/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:53.138850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c6d74-184e-4b8e-807f-153ffc980f97 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:53.238456 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c6d74-184e-4b8e-807f-153ffc980f97 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:53.239998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:53.281367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736b7f8-2b7d-48a2-8b1e-8fe939c249bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:53.282025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736b7f8-2b7d-48a2-8b1e-8fe939c249bf None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:53.282338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736b7f8-2b7d-48a2-8b1e-8fe939c249bf None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:53.282615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736b7f8-2b7d-48a2-8b1e-8fe939c249bf None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:53.283328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736b7f8-2b7d-48a2-8b1e-8fe939c249bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:53.284778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736b7f8-2b7d-48a2-8b1e-8fe939c249bf None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:53.428465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 572/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:36:53 2022] POST /identity/v3/projects => generated 311 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:55.638418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de019bb6-4616-4c8f-80cb-eb69305b0d5d None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:55.639042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de019bb6-4616-4c8f-80cb-eb69305b0d5d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:55.639042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de019bb6-4616-4c8f-80cb-eb69305b0d5d None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:55.640308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 563/1135] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:55.645503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f9cbd-f4ae-4b7e-a02a-0dbb6c72669f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:55.645898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f9cbd-f4ae-4b7e-a02a-0dbb6c72669f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:55.646116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f9cbd-f4ae-4b7e-a02a-0dbb6c72669f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:55.663894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c23f9cbd-f4ae-4b7e-a02a-0dbb6c72669f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:55.665156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c23f9cbd-f4ae-4b7e-a02a-0dbb6c72669f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:55.730919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 573/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:36:55.757060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de019bb6-4616-4c8f-80cb-eb69305b0d5d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:55.799306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:55.863375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de019bb6-4616-4c8f-80cb-eb69305b0d5d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:55.896352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c6221-8564-4b53-a5d2-18fd15f63390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:55.897743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c6221-8564-4b53-a5d2-18fd15f63390 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:55.898022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c6221-8564-4b53-a5d2-18fd15f63390 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:55.898328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c6221-8564-4b53-a5d2-18fd15f63390 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:55.899486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c6221-8564-4b53-a5d2-18fd15f63390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:55.900907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c6221-8564-4b53-a5d2-18fd15f63390 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:55.902543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 564/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:36:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:55.908420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f9cbd-f4ae-4b7e-a02a-0dbb6c72669f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:55.990719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:56.006859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f9cbd-f4ae-4b7e-a02a-0dbb6c72669f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:56.041603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca587184-8ead-4dd1-aa2f-72d91c647958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:56.042287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587184-8ead-4dd1-aa2f-72d91c647958 None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:56.042544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587184-8ead-4dd1-aa2f-72d91c647958 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:56.042948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587184-8ead-4dd1-aa2f-72d91c647958 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:56.043504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca587184-8ead-4dd1-aa2f-72d91c647958 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:56.044970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca587184-8ead-4dd1-aa2f-72d91c647958 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:56.124223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 574/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:36:55 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:36:58.322304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0740-3207-42ba-b708-33316fd99081 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:58.322840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0740-3207-42ba-b708-33316fd99081 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:58.322840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0740-3207-42ba-b708-33316fd99081 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:58.324225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 565/1139] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:36:58.329563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a363a08-c7ae-4073-83af-5be4925c92fd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:58.329996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a363a08-c7ae-4073-83af-5be4925c92fd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:58.330096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a363a08-c7ae-4073-83af-5be4925c92fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:58.351813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-4a363a08-c7ae-4073-83af-5be4925c92fd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:36:58.353007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4a363a08-c7ae-4073-83af-5be4925c92fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:36:58.418929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 575/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:36:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:36:58.443030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0740-3207-42ba-b708-33316fd99081 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:58.545522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0740-3207-42ba-b708-33316fd99081 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:58.557140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:58.584758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:58.585472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:58.585755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:58.586028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:58.596403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:36:58.604083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:36:58.605349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:36:58.606698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 566/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:36:58.611291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a363a08-c7ae-4073-83af-5be4925c92fd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:58.681070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:58.710451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a363a08-c7ae-4073-83af-5be4925c92fd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:58.743785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:58.744494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:58.744775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:58.745046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:58.745794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:58.746100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:58.747511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:58.757138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 576/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:58.762871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:58.884289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861c9c-9c7a-48cd-9ff0-57c33d157aaa None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:58.920563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:58.921438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:58.921719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:58.921994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] PATH_INFO: `/v3/users/25bb09a3fc644e4abe7c960ce3042977` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:58.936722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] RBAC: Authorizing `identity:get_user(user_id=25bb09a3fc644e4abe7c960ce3042977)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:58.938191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:58.939846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 567/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:36:58 2022] GET /identity/v3/users/25bb09a3fc644e4abe7c960ce3042977 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:58.945613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:59.044678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af1f9f-c1e9-4279-9673-9762a26ab329 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:59.074312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:59.079017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:59.079664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:59.079944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:59.080213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:59.147713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:59.149092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:59.248682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 577/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:36:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:36:59.254562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:59.282896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:59.355034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e177501-6d65-4cbd-9653-cfea3c57b2c4 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:59.388959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49deb35-15b8-42b3-8c98-f8d117cace92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:59.389811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49deb35-15b8-42b3-8c98-f8d117cace92 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:59.390093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49deb35-15b8-42b3-8c98-f8d117cace92 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:59.390394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49deb35-15b8-42b3-8c98-f8d117cace92 None admin] PATH_INFO: `/v3/projects/fea884ce876c4727ac137447ec0384cd` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:59.391548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49deb35-15b8-42b3-8c98-f8d117cace92 None admin] RBAC: Authorizing `identity:get_project(project_id=fea884ce876c4727ac137447ec0384cd)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:59.392971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49deb35-15b8-42b3-8c98-f8d117cace92 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:59.394683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 568/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:36:59 2022] GET /identity/v3/projects/fea884ce876c4727ac137447ec0384cd => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:36:59.401612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:36:59.431511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:36:59.500249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff221c8b-fe9c-4f1d-9672-7f4710c78108 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:36:59.537058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22eed6-7b55-4a53-9379-3eabf9604a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:36:59.537846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22eed6-7b55-4a53-9379-3eabf9604a75 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:36:59.538398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22eed6-7b55-4a53-9379-3eabf9604a75 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:36:59.538695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22eed6-7b55-4a53-9379-3eabf9604a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:36:59.539529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22eed6-7b55-4a53-9379-3eabf9604a75 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=fea884ce876c4727ac137447ec0384cd, user.id=25bb09a3fc644e4abe7c960ce3042977) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:36:59.539840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22eed6-7b55-4a53-9379-3eabf9604a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:36:59.541291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22eed6-7b55-4a53-9379-3eabf9604a75 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:36:59.547100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 578/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:36:59 2022] GET /identity/v3/role_assignments?user.id=25bb09a3fc644e4abe7c960ce3042977&scope.project.id=fea884ce876c4727ac137447ec0384cd&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:01.719069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914e694-d68d-4b91-aa9f-1651b7e2dda0 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:01.719632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914e694-d68d-4b91-aa9f-1651b7e2dda0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:01.719632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914e694-d68d-4b91-aa9f-1651b7e2dda0 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:01.720970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 569/1147] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:01.726547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332662c-e23a-48a2-958a-6e4fc01e3c1b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:01.727014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332662c-e23a-48a2-958a-6e4fc01e3c1b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:01.727113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332662c-e23a-48a2-958a-6e4fc01e3c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:01.743976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e332662c-e23a-48a2-958a-6e4fc01e3c1b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:01.745165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e332662c-e23a-48a2-958a-6e4fc01e3c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:01.813668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 579/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:01.836647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914e694-d68d-4b91-aa9f-1651b7e2dda0 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:01.933809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:01.953567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914e694-d68d-4b91-aa9f-1651b7e2dda0 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:01.994776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:01.995570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:01.995859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:01.996137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:02.006711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:37:02.014460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:37:02.015609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:37:02.016683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 570/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:37:02.022072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332662c-e23a-48a2-958a-6e4fc01e3c1b None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:02.048848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:02.121585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332662c-e23a-48a2-958a-6e4fc01e3c1b None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:02.156336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:02.157083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:02.157363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:02.157640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:02.158422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:02.158731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:02.160113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:02.168646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 580/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:02.174618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:02.282396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eaa80-ff69-4a82-90e0-daa2d25fd57d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:02.317404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:02.318299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:02.319376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:02.319567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] PATH_INFO: `/v3/users/25bb09a3fc644e4abe7c960ce3042977` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:02.320795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] RBAC: Authorizing `identity:get_user(user_id=25bb09a3fc644e4abe7c960ce3042977)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:02.322252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:02.323888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 571/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:37:02 2022] GET /identity/v3/users/25bb09a3fc644e4abe7c960ce3042977 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:02.329853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:02.429307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:02.444799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7469a3-50e7-4f09-86b9-bccfbe653348 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:02.481156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a2c87-45a2-4a90-a7ef-f24c65597334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:02.482235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a2c87-45a2-4a90-a7ef-f24c65597334 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:02.482443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a2c87-45a2-4a90-a7ef-f24c65597334 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:02.482721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a2c87-45a2-4a90-a7ef-f24c65597334 None admin] PATH_INFO: `/v3/projects/fea884ce876c4727ac137447ec0384cd` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:02.484044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a2c87-45a2-4a90-a7ef-f24c65597334 None admin] RBAC: Authorizing `identity:get_project(project_id=fea884ce876c4727ac137447ec0384cd)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:02.485713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a2c87-45a2-4a90-a7ef-f24c65597334 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:02.487487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 581/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:37:02 2022] GET /identity/v3/projects/fea884ce876c4727ac137447ec0384cd => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:02.494430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:02.521175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:02.630320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be6d4c-21e6-497a-96fe-31fe6aad3e95 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:02.675573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194b875-69e9-4f03-bd8f-9d321c85aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:02.676821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194b875-69e9-4f03-bd8f-9d321c85aa41 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:02.677204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194b875-69e9-4f03-bd8f-9d321c85aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:02.677582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194b875-69e9-4f03-bd8f-9d321c85aa41 None admin] PATH_INFO: `/v3/projects/fea884ce876c4727ac137447ec0384cd/users/25bb09a3fc644e4abe7c960ce3042977/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:02.681167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194b875-69e9-4f03-bd8f-9d321c85aa41 None admin] RBAC: Authorizing `identity:create_grant(project_id=fea884ce876c4727ac137447ec0384cd, user_id=25bb09a3fc644e4abe7c960ce3042977, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:02.682595 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194b875-69e9-4f03-bd8f-9d321c85aa41 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:02.838551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 572/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:37:02 2022] PUT /identity/v3/projects/fea884ce876c4727ac137447ec0384cd/users/25bb09a3fc644e4abe7c960ce3042977/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:37:04.997724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1458-d132-43f0-b4da-64b3832a0296 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:04.998424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1458-d132-43f0-b4da-64b3832a0296 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:04.998655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1458-d132-43f0-b4da-64b3832a0296 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.000054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 582/1154] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:05.006805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d616-eeef-458f-a2e0-fd7481bbc08b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.009300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d616-eeef-458f-a2e0-fd7481bbc08b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.009300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d616-eeef-458f-a2e0-fd7481bbc08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.025493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5ce6d616-eeef-458f-a2e0-fd7481bbc08b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:05.026749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5ce6d616-eeef-458f-a2e0-fd7481bbc08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:05.095925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 573/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:05.119448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c1458-d132-43f0-b4da-64b3832a0296 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:05.223799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c1458-d132-43f0-b4da-64b3832a0296 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:05.261337 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:05.262048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.262357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.262632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.274017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:37:05.281639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:37:05.283583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:37:05.285550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 583/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:37:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:37:05.290704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6d616-eeef-458f-a2e0-fd7481bbc08b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:05.305754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:05.396473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6d616-eeef-458f-a2e0-fd7481bbc08b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:05.431067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:05.431744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.432032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.432319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.433075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:05.433387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:05.434811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:05.443312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 574/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:37:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:37:05.449821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:05.468489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:05.557515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c6179-719e-4f26-85fb-931f25df9f55 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:05.592128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:05.592983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.593264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.593538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] PATH_INFO: `/v3/users/25bb09a3fc644e4abe7c960ce3042977` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.594732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] RBAC: Authorizing `identity:get_user(user_id=25bb09a3fc644e4abe7c960ce3042977)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:05.596185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:05.597789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 584/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:37:05 2022] GET /identity/v3/users/25bb09a3fc644e4abe7c960ce3042977 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:05.604413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:05.677612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:05.702128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7408e8-8447-4115-b43f-4bbd2966ddee None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:05.735596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:05.736235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:05.736516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:05.736791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:05.802740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:05.805002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:05.876188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:05.901715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 575/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:37:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:37:05.907777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:05.994253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:06.009309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b86770-67a9-4778-a957-8b81249c2c45 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:06.042795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e05cf-2da2-4a5d-b9ea-25cd0e6903e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:06.043867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e05cf-2da2-4a5d-b9ea-25cd0e6903e6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:06.044485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e05cf-2da2-4a5d-b9ea-25cd0e6903e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:06.044763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e05cf-2da2-4a5d-b9ea-25cd0e6903e6 None admin] PATH_INFO: `/v3/projects/fea884ce876c4727ac137447ec0384cd` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:06.045942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e05cf-2da2-4a5d-b9ea-25cd0e6903e6 None admin] RBAC: Authorizing `identity:get_project(project_id=fea884ce876c4727ac137447ec0384cd)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:06.047417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e05cf-2da2-4a5d-b9ea-25cd0e6903e6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:06.049114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 585/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:37:05 2022] GET /identity/v3/projects/fea884ce876c4727ac137447ec0384cd => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:06.055453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:06.152012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6683a4b-752d-470d-856b-76e354a32f72 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:06.185016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1905325-e4bc-4036-bea8-ac58a40c3eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:06.185658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1905325-e4bc-4036-bea8-ac58a40c3eb2 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:06.185940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1905325-e4bc-4036-bea8-ac58a40c3eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:06.186239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1905325-e4bc-4036-bea8-ac58a40c3eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:06.187062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1905325-e4bc-4036-bea8-ac58a40c3eb2 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=fea884ce876c4727ac137447ec0384cd, user.id=25bb09a3fc644e4abe7c960ce3042977) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:06.187377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1905325-e4bc-4036-bea8-ac58a40c3eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:06.188802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1905325-e4bc-4036-bea8-ac58a40c3eb2 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:06.194501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 576/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:37:06 2022] GET /identity/v3/role_assignments?user.id=25bb09a3fc644e4abe7c960ce3042977&scope.project.id=fea884ce876c4727ac137447ec0384cd&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:08.368238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb538c-58db-45f6-98a6-deba6137a6d1 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:08.368828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb538c-58db-45f6-98a6-deba6137a6d1 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:08.368828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb538c-58db-45f6-98a6-deba6137a6d1 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:08.370143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 586/1162] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:08.375600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aaa1d-3575-4c31-9520-6509e54fe467 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:08.376056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aaa1d-3575-4c31-9520-6509e54fe467 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:08.376147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aaa1d-3575-4c31-9520-6509e54fe467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:08.392825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-aa0aaa1d-3575-4c31-9520-6509e54fe467 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:08.394026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-aa0aaa1d-3575-4c31-9520-6509e54fe467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:08.423941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:08.461970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 577/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:08.485027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb538c-58db-45f6-98a6-deba6137a6d1 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:08.589825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb538c-58db-45f6-98a6-deba6137a6d1 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:08.626317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:08.626884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:08.627169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:08.627445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:08.656359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:08.705400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:08.706891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:08.801288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 587/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:37:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:37:08.808006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0aaa1d-3575-4c31-9520-6509e54fe467 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:08.909544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0aaa1d-3575-4c31-9520-6509e54fe467 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:08.944544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868536e-41fa-4a5a-bf23-fe9218ef840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:08.945344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868536e-41fa-4a5a-bf23-fe9218ef840f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:08.945622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868536e-41fa-4a5a-bf23-fe9218ef840f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:08.945889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868536e-41fa-4a5a-bf23-fe9218ef840f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:08.952425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868536e-41fa-4a5a-bf23-fe9218ef840f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:08.953870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868536e-41fa-4a5a-bf23-fe9218ef840f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-2868536e-41fa-4a5a-bf23-fe9218ef840f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:37:08.969147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:37:08.970342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:37:08.971702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 578/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:37:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:37:08.977083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:09.027794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:09.073582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0919e-a2c0-4c46-9919-e1152590ae7c None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:09.107930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdfa55-6b6c-4ced-8b95-6c5a745d5378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:09.108582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdfa55-6b6c-4ced-8b95-6c5a745d5378 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:09.108866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdfa55-6b6c-4ced-8b95-6c5a745d5378 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:09.109144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdfa55-6b6c-4ced-8b95-6c5a745d5378 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:09.109870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdfa55-6b6c-4ced-8b95-6c5a745d5378 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:09.110203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdfa55-6b6c-4ced-8b95-6c5a745d5378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:09.111584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdfa55-6b6c-4ced-8b95-6c5a745d5378 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:09.118753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 588/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Jun 15 14:37:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:11.251912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258f806-3120-46df-9f0b-c65f59fa608d None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:11.252481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258f806-3120-46df-9f0b-c65f59fa608d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:11.252481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258f806-3120-46df-9f0b-c65f59fa608d None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:11.253803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 579/1167] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:11.259219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d579e8-e8ea-4e05-8418-5d9bbc3fd285 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:11.259649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d579e8-e8ea-4e05-8418-5d9bbc3fd285 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:11.259783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d579e8-e8ea-4e05-8418-5d9bbc3fd285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:11.276802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-21d579e8-e8ea-4e05-8418-5d9bbc3fd285 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:11.278010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-21d579e8-e8ea-4e05-8418-5d9bbc3fd285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:11.344238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 589/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:11.367637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258f806-3120-46df-9f0b-c65f59fa608d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:11.395516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:11.473661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258f806-3120-46df-9f0b-c65f59fa608d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:11.513006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b49490-a85e-43ec-b4e6-9a94bea5981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:11.513591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49490-a85e-43ec-b4e6-9a94bea5981e None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:11.513878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49490-a85e-43ec-b4e6-9a94bea5981e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:11.514154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49490-a85e-43ec-b4e6-9a94bea5981e None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:11.514931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b49490-a85e-43ec-b4e6-9a94bea5981e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:11.516301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b49490-a85e-43ec-b4e6-9a94bea5981e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:11.673212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 580/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Wed Jun 15 14:37:11 2022] POST /identity/v3/domains => generated 267 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:37:13.837947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb550d-d7d4-4caf-9848-2a40927c1373 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:13.838547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb550d-d7d4-4caf-9848-2a40927c1373 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:13.838547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb550d-d7d4-4caf-9848-2a40927c1373 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:13.839847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 590/1170] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:13.844986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719daecf-3987-4e68-ab76-cb58f180e2e4 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:13.845395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719daecf-3987-4e68-ab76-cb58f180e2e4 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:13.845598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719daecf-3987-4e68-ab76-cb58f180e2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:13.862690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-719daecf-3987-4e68-ab76-cb58f180e2e4 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:13.863882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-719daecf-3987-4e68-ab76-cb58f180e2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:13.932710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 581/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:13.955799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb550d-d7d4-4caf-9848-2a40927c1373 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:14.005881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:14.059857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb550d-d7d4-4caf-9848-2a40927c1373 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:14.096717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd704a-7928-463a-accf-187d9fc376ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:14.097802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd704a-7928-463a-accf-187d9fc376ef None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:14.098218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd704a-7928-463a-accf-187d9fc376ef None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:14.098691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd704a-7928-463a-accf-187d9fc376ef None admin] PATH_INFO: `/v3/domains/955bc0fa501d445db2ac9d3c24a59d65` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:14.111827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd704a-7928-463a-accf-187d9fc376ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=955bc0fa501d445db2ac9d3c24a59d65)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:14.113300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd704a-7928-463a-accf-187d9fc376ef None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:14.115037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 591/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Wed Jun 15 14:37:13 2022] GET /identity/v3/domains/955bc0fa501d445db2ac9d3c24a59d65 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:14.121705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719daecf-3987-4e68-ab76-cb58f180e2e4 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:14.140376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:14.221311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719daecf-3987-4e68-ab76-cb58f180e2e4 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:14.254463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5b6af-4009-493f-9c05-1f8d122fa693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:14.255116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5b6af-4009-493f-9c05-1f8d122fa693 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:14.255400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5b6af-4009-493f-9c05-1f8d122fa693 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:14.256225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5b6af-4009-493f-9c05-1f8d122fa693 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:14.257126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5b6af-4009-493f-9c05-1f8d122fa693 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:14.258594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5b6af-4009-493f-9c05-1f8d122fa693 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:14.353472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 582/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:37:14 2022] POST /identity/v3/projects => generated 361 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:37:16.564274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574729-b76b-486c-89d8-14d2e2f1c6a1 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.564777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574729-b76b-486c-89d8-14d2e2f1c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.564887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574729-b76b-486c-89d8-14d2e2f1c6a1 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.566199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 592/1174] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:16.571674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9024e0e-eac6-447f-8df5-4fb6b5606f78 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.572063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9024e0e-eac6-447f-8df5-4fb6b5606f78 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.572235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9024e0e-eac6-447f-8df5-4fb6b5606f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.589519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c9024e0e-eac6-447f-8df5-4fb6b5606f78 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:16.590742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c9024e0e-eac6-447f-8df5-4fb6b5606f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:16.652075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:16.656777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 583/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:16.682577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4574729-b76b-486c-89d8-14d2e2f1c6a1 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:16.795161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4574729-b76b-486c-89d8-14d2e2f1c6a1 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:16.804220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:16.831236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7aac1-3c97-40a5-8787-fa772c42a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:16.832092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7aac1-3c97-40a5-8787-fa772c42a989 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.832371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7aac1-3c97-40a5-8787-fa772c42a989 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.832643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7aac1-3c97-40a5-8787-fa772c42a989 None admin] PATH_INFO: `/v3/domains/955bc0fa501d445db2ac9d3c24a59d65` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.835303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7aac1-3c97-40a5-8787-fa772c42a989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=955bc0fa501d445db2ac9d3c24a59d65)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:16.836758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7aac1-3c97-40a5-8787-fa772c42a989 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:16.838426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 593/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Wed Jun 15 14:37:16 2022] GET /identity/v3/domains/955bc0fa501d445db2ac9d3c24a59d65 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:16.844172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9024e0e-eac6-447f-8df5-4fb6b5606f78 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:16.943712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9024e0e-eac6-447f-8df5-4fb6b5606f78 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:16.978633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6893d65-eac8-4419-84ef-1b3e5adfb253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:16.979310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6893d65-eac8-4419-84ef-1b3e5adfb253 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:16.979592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6893d65-eac8-4419-84ef-1b3e5adfb253 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:16.979867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6893d65-eac8-4419-84ef-1b3e5adfb253 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:16.980645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6893d65-eac8-4419-84ef-1b3e5adfb253 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:16.982034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6893d65-eac8-4419-84ef-1b3e5adfb253 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:17.123636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 584/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:37:16 2022] POST /identity/v3/users => generated 311 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:37:19.332724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8e82a-b262-4e2b-83f2-07e06645151f None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.333509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8e82a-b262-4e2b-83f2-07e06645151f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.333509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8e82a-b262-4e2b-83f2-07e06645151f None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.334962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 594/1178] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:19.340240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a202a-530f-45df-9adb-8523fb5d89dc None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.340650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a202a-530f-45df-9adb-8523fb5d89dc None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.340794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a202a-530f-45df-9adb-8523fb5d89dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.358106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-dc7a202a-530f-45df-9adb-8523fb5d89dc None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:19.359300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dc7a202a-530f-45df-9adb-8523fb5d89dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:19.407253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:19.423258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 585/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:19.446683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8e82a-b262-4e2b-83f2-07e06645151f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:19.547354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8e82a-b262-4e2b-83f2-07e06645151f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:19.580454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:19.581244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.581548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.581822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.592712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:37:19.601004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:37:19.602102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:37:19.603167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 595/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:37:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:37:19.608199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a202a-530f-45df-9adb-8523fb5d89dc None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:19.706379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a202a-530f-45df-9adb-8523fb5d89dc None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:19.742314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:19.742980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.743263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.743538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.744285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:19.744592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:19.745975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:19.754261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 586/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:37:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:19.760963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:19.796059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:19.865742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1acbd-c0bb-4941-8637-4a6d869d90ee None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:19.903885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:19.904760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:19.905042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:19.905317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] PATH_INFO: `/v3/users/2fc6af8b59374a3ba16279464bf6dc1d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:19.918730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2fc6af8b59374a3ba16279464bf6dc1d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:19.920213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:19.921936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 596/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:37:19 2022] GET /identity/v3/users/2fc6af8b59374a3ba16279464bf6dc1d => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:19.927742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:19.981942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:20.033519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e6c12-e8fe-417c-8899-12e29efdd81c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:20.066829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:20.067482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:20.067765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:20.068682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:20.137701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:20.139102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:20.196613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:20.267127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 587/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:37:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:37:20.273557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:20.367482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:20.375799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ced0-0b89-4658-8a41-d90e467da7c6 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:20.409646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e3a8b-21b2-4b7b-b969-bc087912cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:20.410459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3a8b-21b2-4b7b-b969-bc087912cfdb None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:20.410746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3a8b-21b2-4b7b-b969-bc087912cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:20.411023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3a8b-21b2-4b7b-b969-bc087912cfdb None admin] PATH_INFO: `/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:20.412884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e3a8b-21b2-4b7b-b969-bc087912cfdb None admin] RBAC: Authorizing `identity:get_project(project_id=c024eca32dd34786a0a5f9e3e0cf52db)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:20.414353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e3a8b-21b2-4b7b-b969-bc087912cfdb None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:20.416047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 597/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:37:20 2022] GET /identity/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:20.422091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:20.517553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c891-d6ec-4eaf-875b-ae1ef919f3ab None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:20.552464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b587e-8298-40a8-80d2-4f88afce2cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:20.553271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b587e-8298-40a8-80d2-4f88afce2cdf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:20.553550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b587e-8298-40a8-80d2-4f88afce2cdf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:20.553819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b587e-8298-40a8-80d2-4f88afce2cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:20.554692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b587e-8298-40a8-80d2-4f88afce2cdf None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=c024eca32dd34786a0a5f9e3e0cf52db, user.id=2fc6af8b59374a3ba16279464bf6dc1d) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:20.555006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b587e-8298-40a8-80d2-4f88afce2cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:20.556465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b587e-8298-40a8-80d2-4f88afce2cdf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:20.562418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 588/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:37:20 2022] GET /identity/v3/role_assignments?user.id=2fc6af8b59374a3ba16279464bf6dc1d&scope.project.id=c024eca32dd34786a0a5f9e3e0cf52db&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:22.696073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361bc78-f282-4ff2-a85f-cc8517e57372 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:22.696590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361bc78-f282-4ff2-a85f-cc8517e57372 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:22.696590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361bc78-f282-4ff2-a85f-cc8517e57372 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:22.697974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 598/1186] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:22.703197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067888d-ba74-4d71-b861-233bce6f86e3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:22.703617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067888d-ba74-4d71-b861-233bce6f86e3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:22.703756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067888d-ba74-4d71-b861-233bce6f86e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:22.721226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-9067888d-ba74-4d71-b861-233bce6f86e3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:22.722468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9067888d-ba74-4d71-b861-233bce6f86e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:22.730786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:22.789513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 589/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:22.814725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361bc78-f282-4ff2-a85f-cc8517e57372 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:22.917933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361bc78-f282-4ff2-a85f-cc8517e57372 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:22.950980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:22.951945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:22.952225 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:22.952501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:22.966592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:37:22.974678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:37:22.975787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:37:22.976956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 599/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:37:22.982970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067888d-ba74-4d71-b861-233bce6f86e3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:23.079909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067888d-ba74-4d71-b861-233bce6f86e3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:23.112908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:23.113565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:23.113845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:23.114116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:23.115291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:23.115404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:23.116965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:23.125390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 590/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:23.132346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:23.155229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:23.235899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b391dbc-3838-49aa-945b-59be22aa1866 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:23.270446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:23.271902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:23.272181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:23.272452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] PATH_INFO: `/v3/users/2fc6af8b59374a3ba16279464bf6dc1d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:23.273640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] RBAC: Authorizing `identity:get_user(user_id=2fc6af8b59374a3ba16279464bf6dc1d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:23.275342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:23.277041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 600/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:37:23 2022] GET /identity/v3/users/2fc6af8b59374a3ba16279464bf6dc1d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:23.283705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:23.288911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:23.387028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293932a-4020-442c-a79b-ae06497abcd6 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:23.419471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14c4f8-e437-492f-a682-d6e902cc570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:23.420324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c4f8-e437-492f-a682-d6e902cc570e None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:23.420606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c4f8-e437-492f-a682-d6e902cc570e None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:23.420879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c4f8-e437-492f-a682-d6e902cc570e None admin] PATH_INFO: `/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:23.422047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14c4f8-e437-492f-a682-d6e902cc570e None admin] RBAC: Authorizing `identity:get_project(project_id=c024eca32dd34786a0a5f9e3e0cf52db)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:23.423609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14c4f8-e437-492f-a682-d6e902cc570e None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:23.425283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 591/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:37:23 2022] GET /identity/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:23.431888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:23.516946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:23.528713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2a061-1e15-4ad9-aa98-6023510de284 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:23.562706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57a0ea-569b-430a-9cb5-a922e525fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:23.563590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57a0ea-569b-430a-9cb5-a922e525fc12 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:23.563864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57a0ea-569b-430a-9cb5-a922e525fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:23.564131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57a0ea-569b-430a-9cb5-a922e525fc12 None admin] PATH_INFO: `/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db/users/2fc6af8b59374a3ba16279464bf6dc1d/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:23.566125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57a0ea-569b-430a-9cb5-a922e525fc12 None admin] RBAC: Authorizing `identity:create_grant(project_id=c024eca32dd34786a0a5f9e3e0cf52db, user_id=2fc6af8b59374a3ba16279464bf6dc1d, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:23.567594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57a0ea-569b-430a-9cb5-a922e525fc12 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:23.625994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 601/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:37:23 2022] PUT /identity/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db/users/2fc6af8b59374a3ba16279464bf6dc1d/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:37:25.738291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54031039-444e-4bac-a2ee-c2c5a8e3b042 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:25.738814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54031039-444e-4bac-a2ee-c2c5a8e3b042 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:25.738814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54031039-444e-4bac-a2ee-c2c5a8e3b042 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:25.740192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 592/1193] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:37:25.745278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0612bf-4ed8-467b-8248-5a6af9183f20 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:25.745702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0612bf-4ed8-467b-8248-5a6af9183f20 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:25.745830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0612bf-4ed8-467b-8248-5a6af9183f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:25.762899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3f0612bf-4ed8-467b-8248-5a6af9183f20 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:37:25.764072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3f0612bf-4ed8-467b-8248-5a6af9183f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:37:25.831335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 602/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:37:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:37:25.855134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54031039-444e-4bac-a2ee-c2c5a8e3b042 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:25.923353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:25.952590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54031039-444e-4bac-a2ee-c2c5a8e3b042 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:25.989168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:25.990152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:25.990468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:25.990743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.001119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:37:26.009562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:37:26.010620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:37:26.011547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 593/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:37:26.016793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0612bf-4ed8-467b-8248-5a6af9183f20 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:26.092855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:26.113155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0612bf-4ed8-467b-8248-5a6af9183f20 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:26.146740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:26.147582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.147867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.148139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.148877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:26.149179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:26.150576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:26.158920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 603/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:26.164374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:26.265136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77175b0-a593-4119-9f7f-3408db36b00d None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:26.293150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:26.297941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:26.298839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.299124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.299400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] PATH_INFO: `/v3/users/2fc6af8b59374a3ba16279464bf6dc1d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.300582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] RBAC: Authorizing `identity:get_user(user_id=2fc6af8b59374a3ba16279464bf6dc1d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:26.302031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:26.304208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 594/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Jun 15 14:37:26 2022] GET /identity/v3/users/2fc6af8b59374a3ba16279464bf6dc1d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:26.310535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:26.417009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29679e64-1ad6-474e-a914-147b72dd1b93 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:26.450058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:26.450736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.451022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.451301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.462408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:26.524440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:26.525936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:26.624412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 604/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:37:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:37:26.630895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:26.729506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aca3b-0923-4fb5-87dd-fe019648a1ff None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:26.762101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea122a4-8369-4951-9fae-f15d44fedd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:26.762985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea122a4-8369-4951-9fae-f15d44fedd14 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.763266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea122a4-8369-4951-9fae-f15d44fedd14 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.763539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea122a4-8369-4951-9fae-f15d44fedd14 None admin] PATH_INFO: `/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.764706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea122a4-8369-4951-9fae-f15d44fedd14 None admin] RBAC: Authorizing `identity:get_project(project_id=c024eca32dd34786a0a5f9e3e0cf52db)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:26.766136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea122a4-8369-4951-9fae-f15d44fedd14 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:26.767849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 595/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Jun 15 14:37:26 2022] GET /identity/v3/projects/c024eca32dd34786a0a5f9e3e0cf52db => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:26.774188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:37:26.816180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:26.874243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7eca27-3ff4-47a3-9cc8-d3a7b760eaab None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:37:26.906901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8398e8f-949e-458c-946a-94127089af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:37:26.907547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8398e8f-949e-458c-946a-94127089af64 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:37:26.907828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8398e8f-949e-458c-946a-94127089af64 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:37:26.908102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8398e8f-949e-458c-946a-94127089af64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:37:26.908949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8398e8f-949e-458c-946a-94127089af64 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=c024eca32dd34786a0a5f9e3e0cf52db, user.id=2fc6af8b59374a3ba16279464bf6dc1d) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:37:26.909270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8398e8f-949e-458c-946a-94127089af64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:37:26.910738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8398e8f-949e-458c-946a-94127089af64 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:37:26.917459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 605/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:37:26 2022] GET /identity/v3/role_assignments?user.id=2fc6af8b59374a3ba16279464bf6dc1d&scope.project.id=c024eca32dd34786a0a5f9e3e0cf52db&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:37:36.757998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=66 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:37:36.914730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=48 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:48.547920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058798e4-b79a-4b3f-b46a-6c0a93f94b7c None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:48.548530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058798e4-b79a-4b3f-b46a-6c0a93f94b7c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:48.548530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058798e4-b79a-4b3f-b46a-6c0a93f94b7c None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:48.549850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 596/1201] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:38:48.555289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1df91-1107-4469-a9af-2fc3605cf776 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:48.555775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1df91-1107-4469-a9af-2fc3605cf776 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:48.555896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1df91-1107-4469-a9af-2fc3605cf776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:48.572739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-86f1df91-1107-4469-a9af-2fc3605cf776 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:48.578223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-86f1df91-1107-4469-a9af-2fc3605cf776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:48.641388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 606/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:38:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:38:48.666089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058798e4-b79a-4b3f-b46a-6c0a93f94b7c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:48.770282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058798e4-b79a-4b3f-b46a-6c0a93f94b7c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:48.808008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:48.809407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:48.809693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:48.809967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:48.820513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:48.821971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:38:48.842646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:48.843840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:48.844817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:38:48.844817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:38:48.844817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:48.844817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:38:48.844817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 597/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:48.850027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1df91-1107-4469-a9af-2fc3605cf776 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:48.953244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1df91-1107-4469-a9af-2fc3605cf776 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:48.983711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:48.990398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86ea43-3b52-4815-83ad-cb0e9fb78317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:48.991088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86ea43-3b52-4815-83ad-cb0e9fb78317 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:48.991362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86ea43-3b52-4815-83ad-cb0e9fb78317 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:48.991781 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86ea43-3b52-4815-83ad-cb0e9fb78317 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:48.992433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86ea43-3b52-4815-83ad-cb0e9fb78317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:48.992744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86ea43-3b52-4815-83ad-cb0e9fb78317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:48.994209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86ea43-3b52-4815-83ad-cb0e9fb78317 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:49.005970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 607/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:49.012456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:49.069042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:49.121474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc15d4-8dcc-46eb-aa66-baf844992265 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:49.163138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5247672-134e-44d4-8e41-0ade5ebda545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:49.163852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5247672-134e-44d4-8e41-0ade5ebda545 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:49.164131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5247672-134e-44d4-8e41-0ade5ebda545 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:49.164406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5247672-134e-44d4-8e41-0ade5ebda545 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:49.165138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5247672-134e-44d4-8e41-0ade5ebda545 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:49.166735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5247672-134e-44d4-8e41-0ade5ebda545 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:49.249950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 598/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Jun 15 14:38:49 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:38:51.436423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbffdb-accd-49aa-aa43-626fe413fd4d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:51.436984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbffdb-accd-49aa-aa43-626fe413fd4d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:51.436984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbffdb-accd-49aa-aa43-626fe413fd4d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:51.438535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 608/1206] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:38:51.444182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbaa78-4dcd-45c9-99bf-ac4b118ff075 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:51.444632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbaa78-4dcd-45c9-99bf-ac4b118ff075 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:51.444734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbaa78-4dcd-45c9-99bf-ac4b118ff075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:51.462924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4dbbaa78-4dcd-45c9-99bf-ac4b118ff075 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:51.464110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4dbbaa78-4dcd-45c9-99bf-ac4b118ff075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:51.534457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 599/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:38:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 14:38:51.561220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbffdb-accd-49aa-aa43-626fe413fd4d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:51.663609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbffdb-accd-49aa-aa43-626fe413fd4d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:51.700106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:51.700870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:51.701168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:51.701420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:51.712777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:38:51.720721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:51.721904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:51.722793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 609/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:51.728339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbaa78-4dcd-45c9-99bf-ac4b118ff075 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:51.779706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:51.844974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbaa78-4dcd-45c9-99bf-ac4b118ff075 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:51.880396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:51.881072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:51.881437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:51.881715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:51.882499 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:51.882809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:51.884811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:51.893335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 600/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:38:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:51.899184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:51.951917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:52.000525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33d9ee-01ca-47a9-96ce-072d00c69a0a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:52.035375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:52.036164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:52.036442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:52.036713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:52.047025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:52.048595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:38:52.068670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:52.069793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:52.070799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:38:52.070799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:38:52.070799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:52.070799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:52.070799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 610/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:52.076796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:52.177398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:52.181334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f655c-7d89-457c-aac0-7d30d8fa1f1c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:52.219134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:52.219717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:52.220016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:52.220274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:52.220996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:52.221303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:52.222733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:52.234452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 601/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:52.240239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:52.343399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f1f62-2c9d-46bc-b74c-5b01907c0c05 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:52.344984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:52.379566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:52.380379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:52.380664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:52.380938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:52.389176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:52.390760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:38:52.406555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:52.408138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:52.409420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 611/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:38:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:52.413856 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:52.519502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27553560-4b91-46ce-ae38-e33549c9c74f None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:52.557858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:52.560211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:52.560211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:52.560211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:52.560648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:52.560960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:52.562418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:52.581230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 602/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Jun 15 14:38:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:52.587282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:52.689475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4629d11-5eab-4c3a-8296-05ccdb0b2e88 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:52.722936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:52.723592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:52.723880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:52.724154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:52.766920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:52.805128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:52.806548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:52.906956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 612/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:38:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:38:52.912982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:52.934248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:53.016865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb600-4c1f-4507-9aa9-2a2d0f832595 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:53.052154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:53.052982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:53.053360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:53.057660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:53.064544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:53.066041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:38:53.084841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:53.085974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:53.087436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:38:53.087436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:38:53.087436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:53.087436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:38:53.087436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 603/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:53.092777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:53.154915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:53.193587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f406c-b56d-41d9-9332-3b53e769bc28 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:53.229485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:53.230154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:53.230519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:53.231520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:53.232322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:53.232631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:53.234061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:53.250937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 613/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:53.256782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:53.339191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:53.373533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f504-8a2f-4e57-8b51-cbeaebb4b83a None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:53.414086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:53.414933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:53.415228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:53.415509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:53.422514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:53.423969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:38:53.439543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:53.444444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:38:53.445519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 604/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:53.447403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:53.554422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd72c5c-c1c7-47a4-a7d7-5ee199946d37 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:53.577457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:53.589347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcf26d-b80c-41f9-9463-dd88aaf0ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:53.590761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf26d-b80c-41f9-9463-dd88aaf0ee1a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:53.590936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf26d-b80c-41f9-9463-dd88aaf0ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:53.591208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf26d-b80c-41f9-9463-dd88aaf0ee1a None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:53.591999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcf26d-b80c-41f9-9463-dd88aaf0ee1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:53.592309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcf26d-b80c-41f9-9463-dd88aaf0ee1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:53.593754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcf26d-b80c-41f9-9463-dd88aaf0ee1a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:53.605988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 614/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:53.612250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:53.725576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cdf2c-8cad-4507-9904-8cdceb261dfd None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:53.758586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:53.767503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2364fd6-b61c-4917-bd41-ed62c8902fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:53.768325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364fd6-b61c-4917-bd41-ed62c8902fc6 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:53.768608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364fd6-b61c-4917-bd41-ed62c8902fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:53.768886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364fd6-b61c-4917-bd41-ed62c8902fc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:53.769891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364fd6-b61c-4917-bd41-ed62c8902fc6 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=6998e874e8264977a7dc412436162a02) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:53.770334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364fd6-b61c-4917-bd41-ed62c8902fc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:53.771792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364fd6-b61c-4917-bd41-ed62c8902fc6 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:53.777892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 605/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:38:53 2022] GET /identity/v3/role_assignments?user.id=6998e874e8264977a7dc412436162a02&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:55.947815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d332e94-61cc-4aa4-8a75-2fcb256ac1d1 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:55.948570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d332e94-61cc-4aa4-8a75-2fcb256ac1d1 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:55.948570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d332e94-61cc-4aa4-8a75-2fcb256ac1d1 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:55.950403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 615/1220] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:38:55.956072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c134-5383-46b0-9f99-39b32a1b000a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:55.956693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c134-5383-46b0-9f99-39b32a1b000a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:55.956693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c134-5383-46b0-9f99-39b32a1b000a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:55.975201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5a19c134-5383-46b0-9f99-39b32a1b000a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:38:55.976403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5a19c134-5383-46b0-9f99-39b32a1b000a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:38:56.044628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 606/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:38:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:38:56.067623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d332e94-61cc-4aa4-8a75-2fcb256ac1d1 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.184528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d332e94-61cc-4aa4-8a75-2fcb256ac1d1 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.220482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.221242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.221524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.221799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.232237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:38:56.240189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:56.241334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:56.242426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 616/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:56.248214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19c134-5383-46b0-9f99-39b32a1b000a None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.349382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19c134-5383-46b0-9f99-39b32a1b000a None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.374702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:56.385064 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.385732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.386015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.386325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.387111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] RBAC: Adding query filter params (name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:56.387416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.388784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.396944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 607/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:56.403679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.450670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:56.508641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce26d7-c309-47bc-9b77-668e5caf02f4 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.541980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.542985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.543291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.543569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.554871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.556451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:38:56.573608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:56.575164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:56.576262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:38:56.576262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:38:56.576262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:56.576262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:56.576262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 617/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:56.581404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.688822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28224a7c-4660-4c48-a2b0-d2a6fd26ed91 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.726291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.727447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.727967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.728493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.729245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:56.729644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.731553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.747442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 608/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:56.753478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.860846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:56.867316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5ee9e-a32a-43df-9a8e-a019718b2386 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:56.901665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:56.902564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:56.902887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:56.903167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:56.911199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:56.922301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:38:56.927057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:56.928764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:56.930192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 618/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:56.934893 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:56.984514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:57.038426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56798337-96dd-414b-bc5d-e3c87802b9f8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.083011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.083688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.083991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.084269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.085113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:57.085424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.088661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:57.107460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 609/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Jun 15 14:38:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:57.113918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:57.216419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98153da-85cc-4ba1-839b-383939124b59 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.251133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.251925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.252202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.252470 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.263591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.265065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:38:57.282080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:57.283189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:57.284319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:38:57.284319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:38:57.284319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:38:57.284319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:57.284319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 619/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:57.289336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:57.391978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c1578-4c9d-465c-8eac-de634a34fb39 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.394629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:57.427423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.428089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.428374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.428650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.429377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:57.429686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.431107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:57.443374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 610/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:57.449436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:57.460487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:57.554598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9c44a-1454-4138-a9e9-2bc5bac7341f None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.589292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.590660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.590945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.591221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.598050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.599540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:38:57.614745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:38:57.615875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:38:57.616992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 620/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:38:57.621991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:57.724518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92598-6962-47d7-86b4-0d8cbb83d0cf None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.760500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcad9c-04b9-479b-a956-72c9163124a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.761160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcad9c-04b9-479b-a956-72c9163124a3 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.761468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcad9c-04b9-479b-a956-72c9163124a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.761936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcad9c-04b9-479b-a956-72c9163124a3 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.762841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcad9c-04b9-479b-a956-72c9163124a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:38:57.763155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcad9c-04b9-479b-a956-72c9163124a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.764600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcad9c-04b9-479b-a956-72c9163124a3 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:57.777024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 611/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:38:57.784420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:38:57.854290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:38:57.889492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8e303-1fff-4ec3-90d6-6c8f1e92d844 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:38:57.923987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62be97d-8d7b-4c82-815a-43b78f176ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:38:57.924879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62be97d-8d7b-4c82-815a-43b78f176ba9 None admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:38:57.925160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62be97d-8d7b-4c82-815a-43b78f176ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:38:57.925436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62be97d-8d7b-4c82-815a-43b78f176ba9 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/6998e874e8264977a7dc412436162a02/roles/44450886f4e74405a0d654944c8740d4` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:38:57.941048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62be97d-8d7b-4c82-815a-43b78f176ba9 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=6998e874e8264977a7dc412436162a02, role_id=44450886f4e74405a0d654944c8740d4)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:38:57.942568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62be97d-8d7b-4c82-815a-43b78f176ba9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:38:58.026319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 621/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:38:57 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/6998e874e8264977a7dc412436162a02/roles/44450886f4e74405a0d654944c8740d4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:39:00.163542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927406ac-194b-4ec8-ba1c-36cd406b6b4d None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:00.164125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927406ac-194b-4ec8-ba1c-36cd406b6b4d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:00.164125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927406ac-194b-4ec8-ba1c-36cd406b6b4d None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:00.165438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 612/1233] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:00.171340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d354dd-9dc1-48f0-ba1d-991a693c6750 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:00.171783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d354dd-9dc1-48f0-ba1d-991a693c6750 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:00.171878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d354dd-9dc1-48f0-ba1d-991a693c6750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:00.188937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-07d354dd-9dc1-48f0-ba1d-991a693c6750 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:00.190134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-07d354dd-9dc1-48f0-ba1d-991a693c6750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:00.258504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 622/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:00.282550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927406ac-194b-4ec8-ba1c-36cd406b6b4d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:00.309998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:00.393239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927406ac-194b-4ec8-ba1c-36cd406b6b4d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:00.431378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:00.432082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:00.432367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:00.432648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:00.443171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:39:00.451858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:00.456007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:00.457777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 613/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Jun 15 14:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:00.459479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d354dd-9dc1-48f0-ba1d-991a693c6750 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:00.507658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:00.567244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d354dd-9dc1-48f0-ba1d-991a693c6750 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:00.602239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:00.602913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:00.603195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:00.603469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:00.604209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] RBAC: Adding query filter params (name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:00.604516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:00.606232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:00.615281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 623/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Jun 15 14:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:39:00.621178 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:00.706742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:00.723531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6139-82e6-403b-9d6e-590de7fdce13 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:00.759403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:00.760200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:00.760483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:00.760760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:00.771048 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:00.772523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:00.790298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:00.796586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:00.797761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:00.797761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:00.797761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:00.797761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:00.797761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 614/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:00.798412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:00.906701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62785cf8-74a6-4f84-8c84-0e1540c5af74 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:00.916130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:00.941433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:00.942096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:00.942405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:00.942685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:00.943403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:00.943705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:00.945100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:00.957325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 624/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:00.963374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:01.076131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33526c-9f11-417a-91ce-51e073dd7be7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:01.123521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:01.124332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:01.124607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:01.124879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:01.127939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:01.132656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:01.134111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:39:01.149900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:01.153005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 615/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:01.162074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:01.263124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda87b5-a2d0-4aa5-83fa-4aa1a5900ffa None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:01.297803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:01.298506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:01.298791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:01.299067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:01.299906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:01.300218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:01.301660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:01.319464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 625/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Jun 15 14:39:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:01.326257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:01.444407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2152-70bd-4924-b8df-8b5cbb88b1d5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:01.482815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:01.483470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:01.483755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:01.484029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:01.545894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:01.552390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:01.554026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:01.652532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 616/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:39:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 14:39:01.658768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:01.686990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:01.763132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89ff5e-24d4-4592-a336-1bcacfdfaa5a None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:01.796691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:01.797384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:01.797659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:01.797925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:01.808189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:01.810221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:01.828612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:01.829695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:01.835148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:01.835148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:01.835148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:01.835148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:01.835148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 626/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:01.835776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:01.934967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:01.940820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edab313-056c-447f-ba4f-c48a4b739472 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:01.975159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:01.981852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:01.981852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:01.981852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:01.981852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:01.981852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:01.981852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:01.991224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 617/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:01.998029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:02.079397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:02.101220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f4f6-d9fd-4020-9b39-236cf62faaff None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:02.135261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:02.136056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:02.136343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:02.137155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:02.144086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:02.145709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:39:02.160209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:02.161317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:02.162498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 627/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:39:02.171728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:02.273668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304231c-2232-4b73-a743-abf0d18af276 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:02.308183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e339581-fa6d-478e-b21f-da7f073c09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:02.309472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339581-fa6d-478e-b21f-da7f073c09dd None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:02.309754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339581-fa6d-478e-b21f-da7f073c09dd None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:02.310027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339581-fa6d-478e-b21f-da7f073c09dd None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:02.310857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e339581-fa6d-478e-b21f-da7f073c09dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:02.311165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e339581-fa6d-478e-b21f-da7f073c09dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:02.312592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e339581-fa6d-478e-b21f-da7f073c09dd None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:02.325636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 618/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:02.331847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:02.438343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c09a0-b3e3-43a6-a388-21a182e5c87e None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:02.473333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec96d5-8a34-4eb1-95e8-99504e9ce016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:02.473998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec96d5-8a34-4eb1-95e8-99504e9ce016 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:02.474313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec96d5-8a34-4eb1-95e8-99504e9ce016 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:02.474591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec96d5-8a34-4eb1-95e8-99504e9ce016 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:02.475428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec96d5-8a34-4eb1-95e8-99504e9ce016 None admin] RBAC: Adding query filter params (role.id=44450886f4e74405a0d654944c8740d4, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=6998e874e8264977a7dc412436162a02) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:02.475739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec96d5-8a34-4eb1-95e8-99504e9ce016 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:02.477208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec96d5-8a34-4eb1-95e8-99504e9ce016 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:02.483712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 628/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:39:02 2022] GET /identity/v3/role_assignments?user.id=6998e874e8264977a7dc412436162a02&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=44450886f4e74405a0d654944c8740d4 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:04.745758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c061b5-44f8-4884-9fdb-ebfba9dfbdd9 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:04.747104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c061b5-44f8-4884-9fdb-ebfba9dfbdd9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:04.747104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c061b5-44f8-4884-9fdb-ebfba9dfbdd9 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:04.748232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 619/1247] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:04.753922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a904d1-8903-4efd-b1fc-1cbde24708f6 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:04.754414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a904d1-8903-4efd-b1fc-1cbde24708f6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:04.756278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a904d1-8903-4efd-b1fc-1cbde24708f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:04.759560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:04.772263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d1a904d1-8903-4efd-b1fc-1cbde24708f6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:04.773451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d1a904d1-8903-4efd-b1fc-1cbde24708f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:04.841914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 629/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:04.866289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c061b5-44f8-4884-9fdb-ebfba9dfbdd9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:04.886881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:04.974991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c061b5-44f8-4884-9fdb-ebfba9dfbdd9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:05.010121 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:05.010948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:05.011235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:05.011568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:05.022054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:39:05.029928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:05.031087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:05.032102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 620/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:05.037585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a904d1-8903-4efd-b1fc-1cbde24708f6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:05.141153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a904d1-8903-4efd-b1fc-1cbde24708f6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:05.174975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:05.181203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:05.181892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:05.182190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:05.182480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:05.183245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:05.184127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:05.185626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:05.194601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 630/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:39:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:05.201401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:05.291753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:05.315067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a68ef-d50e-4084-bbbc-75bb009fe8d7 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:05.350663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:05.351461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:05.351742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:05.352017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:05.363919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:05.372872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:05.382836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:05.383927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:05.385029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:05.385029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:05.385029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:05.385029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:05.385029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 621/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:39:05.390231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:05.494386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f990bd-1fa8-4544-8cd0-9144110cadd9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:05.537099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:05.537783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:05.538059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:05.538414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:05.539217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:05.539628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:05.541045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:05.553775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 631/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:05.560771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:05.664179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3267d1-9ff3-4bfc-9436-ecae193c96d1 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:05.689919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:05.697431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:05.698259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:05.698546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:05.698825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:05.706893 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:05.708334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:39:05.722973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:05.724162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:05.725455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:05.726547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 622/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:05.730688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:05.788144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:05.834653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c7a41-8091-4914-89df-7ae8ef03f963 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:05.870185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:05.871511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:05.871804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:05.872079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:05.872922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:05.873234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:05.874734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:05.895172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 632/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Jun 15 14:39:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:05.901260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:06.011120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca947b-a48f-4166-9476-39bf1bdc8f33 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:06.048561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:06.049138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:06.049415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:06.049687 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:06.107680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:06.119512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:06.121084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:06.224075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 623/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:39:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:39:06.231243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:06.332692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ab0a6-733d-4471-b2ab-bbf70eda985d None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:06.348339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:06.370290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:06.371373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:06.371813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:06.372232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:06.382604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:06.384097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:06.401944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:06.403988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:06.405096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:06.405096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:06.405096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:06.405096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:06.405376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 633/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:06.410849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:06.503572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:06.519799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82444d89-913c-4dde-8baa-8979f18b5e2c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:06.566660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:06.567859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:06.568139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:06.568413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:06.569141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:06.571142 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:06.572530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:06.587460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 624/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:06.593849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:06.697116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6e82-2b89-4b18-8563-14053e828867 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:06.732528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:06.733326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:06.733609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:06.733885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:06.740717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:06.742202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:39:06.756577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:06.758249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:06.759211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 634/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:06.764635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:06.868328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b396e1-658d-4855-a546-c6f8bbf91c52 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:06.903489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edcbf2-e79f-4473-9f84-f95a7e57d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:06.904201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edcbf2-e79f-4473-9f84-f95a7e57d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:06.904427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edcbf2-e79f-4473-9f84-f95a7e57d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:06.904703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edcbf2-e79f-4473-9f84-f95a7e57d9c4 None admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:06.905489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edcbf2-e79f-4473-9f84-f95a7e57d9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:06.905798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edcbf2-e79f-4473-9f84-f95a7e57d9c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:06.907260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edcbf2-e79f-4473-9f84-f95a7e57d9c4 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:06.920282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 625/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:06.926613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:06.952880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:07.026603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a6259-46ca-49f6-8e3f-d76e29c865e2 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:07.061651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4adc0-cc7a-43cc-b7be-99e75ca5caad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:07.062339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4adc0-cc7a-43cc-b7be-99e75ca5caad None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:07.063163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4adc0-cc7a-43cc-b7be-99e75ca5caad None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:07.063440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4adc0-cc7a-43cc-b7be-99e75ca5caad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:07.064279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4adc0-cc7a-43cc-b7be-99e75ca5caad None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=6998e874e8264977a7dc412436162a02) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:07.064593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4adc0-cc7a-43cc-b7be-99e75ca5caad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:07.066092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4adc0-cc7a-43cc-b7be-99e75ca5caad None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:07.072005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 635/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:39:06 2022] GET /identity/v3/role_assignments?user.id=6998e874e8264977a7dc412436162a02&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:09.234934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c486c3-adee-4b3f-8028-2b95a866617c None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:09.235500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c486c3-adee-4b3f-8028-2b95a866617c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:09.235500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c486c3-adee-4b3f-8028-2b95a866617c None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:09.236824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 626/1261] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:09.242332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd87c1-085d-4758-aa11-5928d01ce9f3 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:09.242768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd87c1-085d-4758-aa11-5928d01ce9f3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:09.242886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd87c1-085d-4758-aa11-5928d01ce9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:09.259897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-53dd87c1-085d-4758-aa11-5928d01ce9f3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:09.261108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-53dd87c1-085d-4758-aa11-5928d01ce9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:09.357829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 636/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:09.381454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c486c3-adee-4b3f-8028-2b95a866617c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:09.392732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:09.497538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c486c3-adee-4b3f-8028-2b95a866617c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:09.536230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:09.537044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:09.537326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:09.537607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:09.548938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:39:09.557250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:09.564294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:09.565144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 627/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:09.565841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd87c1-085d-4758-aa11-5928d01ce9f3 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:09.584208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:09.670452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd87c1-085d-4758-aa11-5928d01ce9f3 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:09.707594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:09.709420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:09.709706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:09.709981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:09.710772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:09.711086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:09.713402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:09.731444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 637/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:39:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:39:09.737471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:09.808040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:09.843259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27198f82-012f-4565-aba1-6a76d1928594 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:09.880185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:09.881035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:09.881313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:09.881586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:09.892680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:09.894269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:09.912522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:09.913680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:09.914802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:09.914802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:09.914802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:09.914802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:09.914802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 628/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:09.920332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:10.013616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:10.032682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58596f8-6284-4615-8e5a-b1cc0135318b None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:10.068311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:10.068905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:10.069191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:10.069470 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:10.070216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:10.070529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:10.071917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:10.085172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 638/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:10.091410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:10.200741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561424-d6d8-4ae0-a493-d2142b90e39b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:10.221525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:10.236183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:10.236992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:10.237615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:10.237615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:10.245365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:10.247588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:39:10.262572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:10.265659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:10.266926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 629/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 14:39:10.269796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:10.373736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5d929-483c-4dc3-a65e-7c6532615d57 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:10.409324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:10.410061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:10.410327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:10.410691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:10.412008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:10.412008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:10.413150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:10.422291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:10.433374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 639/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Jun 15 14:39:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:10.439795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:10.554891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebb093-ae21-4374-bea6-a226d12ddaf8 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:10.589312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:10.590705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:10.590985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:10.591255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:10.602688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:10.604165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:10.625547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:10.626580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:10.627461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:10.627461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:10.627461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:10.627461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:10.627461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 630/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:10.632873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:10.740300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977800d-0db6-411f-adf9-c461b8924330 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:10.776475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:10.777307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:10.777624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:10.777889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:10.778777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:10.779090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:10.780486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:10.793716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 640/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:10.799407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:10.837896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:10.908811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4a413-f924-4d4e-b137-acebeb587aa5 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:10.950041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:10.950952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:10.951229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:10.951503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:10.958876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:10.961293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:39:10.975236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:10.976420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:10.977467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 631/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:10.983131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:11.045270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:11.086534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeee007-cc61-4a05-a1aa-ff2047e53390 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:11.119459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe51fa7-7cb9-4e0a-903e-5ca4bbebc76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:11.120117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51fa7-7cb9-4e0a-903e-5ca4bbebc76d None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:11.120396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51fa7-7cb9-4e0a-903e-5ca4bbebc76d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:11.120669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51fa7-7cb9-4e0a-903e-5ca4bbebc76d None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:11.122623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe51fa7-7cb9-4e0a-903e-5ca4bbebc76d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:11.122924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe51fa7-7cb9-4e0a-903e-5ca4bbebc76d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:11.124854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe51fa7-7cb9-4e0a-903e-5ca4bbebc76d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:11.138465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 641/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:11.144659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:11.240200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:11.249401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d066a-73e5-41c8-b70b-39da26115a52 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:11.282378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4622361-1b9b-4240-8733-b3dd17fd15d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:11.283265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4622361-1b9b-4240-8733-b3dd17fd15d9 None admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:11.283542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4622361-1b9b-4240-8733-b3dd17fd15d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:11.283813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4622361-1b9b-4240-8733-b3dd17fd15d9 None admin] PATH_INFO: `/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/6998e874e8264977a7dc412436162a02/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:11.285755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4622361-1b9b-4240-8733-b3dd17fd15d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=abc7363e773e432ea1d89f126f3dcf5e, user_id=6998e874e8264977a7dc412436162a02, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:11.287212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4622361-1b9b-4240-8733-b3dd17fd15d9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:11.387182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 632/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Jun 15 14:39:11 2022] PUT /identity/v3/projects/abc7363e773e432ea1d89f126f3dcf5e/users/6998e874e8264977a7dc412436162a02/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:39:13.563891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9cf23-4eda-4e6b-901f-c4edd355a89d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:13.564450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9cf23-4eda-4e6b-901f-c4edd355a89d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:13.564450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9cf23-4eda-4e6b-901f-c4edd355a89d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:13.565822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 642/1274] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:13.571119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd412e67-4a23-4962-ac2d-978ae90e1d37 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:13.571541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd412e67-4a23-4962-ac2d-978ae90e1d37 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:13.571669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd412e67-4a23-4962-ac2d-978ae90e1d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:13.588418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-dd412e67-4a23-4962-ac2d-978ae90e1d37 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:13.590157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dd412e67-4a23-4962-ac2d-978ae90e1d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:13.658056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 633/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:13.682526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9cf23-4eda-4e6b-901f-c4edd355a89d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:13.786605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9cf23-4eda-4e6b-901f-c4edd355a89d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:13.805259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:13.822160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:13.823035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:13.823316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:13.823589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:13.834037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 14:39:13.842678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:13.845807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:13.847267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 643/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Jun 15 14:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:13.850022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd412e67-4a23-4962-ac2d-978ae90e1d37 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:13.935259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:13.954355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd412e67-4a23-4962-ac2d-978ae90e1d37 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:13.989774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:13.990467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:13.990748 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:13.991019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:13.991761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:13.992066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:13.993437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:14.002400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 634/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Jun 15 14:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:14.008518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:14.118475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23928e-8d54-43f9-9820-a31a92876931 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:14.154379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:14.155182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:14.155468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:14.155741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:14.166483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:14.168182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:14.187750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:14.188920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:14.190027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:14.190027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:14.190027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:14.190027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:14.190027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 644/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:14.194875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:14.298722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b686-da10-49b8-b4a1-750ee01963ba None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:14.333143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:14.333833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:14.334113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:14.334418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:14.335145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:14.335512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:14.336932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:14.350592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 635/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:14.356928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:14.390631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:14.459243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bf4da-151e-4e67-98b4-78c8728c21cd None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:14.493442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:14.494495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:14.494850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:14.495286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:14.504219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:14.506256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 14:39:14.520984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:14.522247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:14.523569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 645/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:14.529822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:14.533770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:14.637289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455a0fa-04d2-4f7f-93f6-03779b323d5b None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:14.672535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:14.673125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:14.673407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:14.673684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:14.674556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:14.674870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:14.676313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:14.697481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 636/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Jun 15 14:39:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:39:14.706149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:14.799923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:14.809014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721900-c695-422b-a1fd-663250f3c285 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:14.843150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:14.843806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:14.844090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:14.844387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:14.912758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:14.914259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:15.006094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:15.017058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 646/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Jun 15 14:39:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 14:39:15.025891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:15.093215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:15.139904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0e036-9a8c-41cf-b6b9-2acb011f7963 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:15.181411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:15.182233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:15.182551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:15.182828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:15.193561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:15.195046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 14:39:15.212746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:15.213950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:15.215816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 14:39:15.215816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 14:39:15.215816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 14:39:15.215816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:15.215816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 637/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:15.220367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:15.321526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14f705-19de-4dee-a9de-4c711280de93 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:15.357369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:15.358029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:15.358343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:15.358618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:15.359344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:15.359644 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:15.361101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:15.376664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 647/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Jun 15 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:15.383152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:15.487503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea01121-ab6f-4c2b-9d28-e80467b4d633 None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:15.501120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:15.523874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:15.524700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:15.525014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:15.525295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:15.533633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:15.535138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 14:39:15.550669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:15.551775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: ERROR keystone.server.flask.application Jun 15 14:39:15.552674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 638/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Jun 15 14:39:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:15.557973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:15.611022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:15.660500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f2447-3b28-4d52-8d91-a204f7231173 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:15.693247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ac9b4-8fda-49ad-9561-df82d0f54e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:15.693986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ac9b4-8fda-49ad-9561-df82d0f54e90 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:15.694273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ac9b4-8fda-49ad-9561-df82d0f54e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:15.697045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ac9b4-8fda-49ad-9561-df82d0f54e90 None admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:15.697045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ac9b4-8fda-49ad-9561-df82d0f54e90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:15.697045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ac9b4-8fda-49ad-9561-df82d0f54e90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:15.697501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ac9b4-8fda-49ad-9561-df82d0f54e90 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:15.711035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 648/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Jun 15 14:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:15.718388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:15.820379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe10da9-2717-4ee7-87f1-15faaf4111fb None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:15.856890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311387cd-498a-4682-adc7-c6875efa70c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:15.857570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311387cd-498a-4682-adc7-c6875efa70c9 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:15.857828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311387cd-498a-4682-adc7-c6875efa70c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:15.858103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311387cd-498a-4682-adc7-c6875efa70c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:15.858967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311387cd-498a-4682-adc7-c6875efa70c9 None admin] RBAC: Adding query filter params (role.id=2ffcf751c1e14a28ad9d1962a027c04b, scope.project.id=abc7363e773e432ea1d89f126f3dcf5e, user.id=6998e874e8264977a7dc412436162a02) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:15.859275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311387cd-498a-4682-adc7-c6875efa70c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:15.860711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311387cd-498a-4682-adc7-c6875efa70c9 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:15.867097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 639/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Jun 15 14:39:15 2022] GET /identity/v3/role_assignments?user.id=6998e874e8264977a7dc412436162a02&scope.project.id=abc7363e773e432ea1d89f126f3dcf5e&role.id=2ffcf751c1e14a28ad9d1962a027c04b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:18.068905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002f85-4227-4ac5-9dd1-3ca5bd80f123 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:18.069482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002f85-4227-4ac5-9dd1-3ca5bd80f123 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:18.069482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002f85-4227-4ac5-9dd1-3ca5bd80f123 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:18.070974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 649/1288] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:18.077099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac11c1-63cb-424c-812d-f5d0cb07ca7f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:18.078768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac11c1-63cb-424c-812d-f5d0cb07ca7f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:18.078768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac11c1-63cb-424c-812d-f5d0cb07ca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:18.095209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c1ac11c1-63cb-424c-812d-f5d0cb07ca7f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:18.096549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c1ac11c1-63cb-424c-812d-f5d0cb07ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:18.127285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:18.165651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 640/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:18.190257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002f85-4227-4ac5-9dd1-3ca5bd80f123 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:18.302308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002f85-4227-4ac5-9dd1-3ca5bd80f123 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:18.313245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:18.340970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:18.341866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:18.342063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:18.342363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:18.343027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:18.344415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:18.356824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:18.357897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:18.358810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 650/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:39:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:18.364067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac11c1-63cb-424c-812d-f5d0cb07ca7f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:18.469354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac11c1-63cb-424c-812d-f5d0cb07ca7f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:18.505964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492efc2-d0f1-4592-9c8f-36ee351ecb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:18.506675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492efc2-d0f1-4592-9c8f-36ee351ecb17 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:18.506954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492efc2-d0f1-4592-9c8f-36ee351ecb17 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:18.507222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492efc2-d0f1-4592-9c8f-36ee351ecb17 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:18.507923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492efc2-d0f1-4592-9c8f-36ee351ecb17 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:18.508225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492efc2-d0f1-4592-9c8f-36ee351ecb17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:18.509585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492efc2-d0f1-4592-9c8f-36ee351ecb17 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:18.515836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 641/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Jun 15 14:39:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:18.521747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:18.630463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4785cfa-7afe-4a71-bf47-4dfe1cacf2b1 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:18.665672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e96e9c-8354-48cf-96f0-960843ad59a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:18.666368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96e9c-8354-48cf-96f0-960843ad59a3 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:18.666646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96e9c-8354-48cf-96f0-960843ad59a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:18.666917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96e9c-8354-48cf-96f0-960843ad59a3 None admin] PATH_INFO: `/v3/services` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:18.667692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e96e9c-8354-48cf-96f0-960843ad59a3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:18.667998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e96e9c-8354-48cf-96f0-960843ad59a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:18.669391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e96e9c-8354-48cf-96f0-960843ad59a3 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:18.674511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 651/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Jun 15 14:39:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:20.842629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd05f4-330f-4992-80e5-b363b25f12f9 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:20.843152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd05f4-330f-4992-80e5-b363b25f12f9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:20.843152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd05f4-330f-4992-80e5-b363b25f12f9 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:20.844440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 642/1293] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:20.849568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c331-6dc5-4da5-bd67-9d327386543b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:20.849993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c331-6dc5-4da5-bd67-9d327386543b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:20.850158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c331-6dc5-4da5-bd67-9d327386543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:20.867948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ea49c331-6dc5-4da5-bd67-9d327386543b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:20.869167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ea49c331-6dc5-4da5-bd67-9d327386543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:20.905474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:20.943624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 652/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:20.967445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd05f4-330f-4992-80e5-b363b25f12f9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:21.002304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:21.075500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd05f4-330f-4992-80e5-b363b25f12f9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:21.111226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb7525-f746-40e0-ba51-37eb3156ef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:21.111782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7525-f746-40e0-ba51-37eb3156ef27 None admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:21.112091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7525-f746-40e0-ba51-37eb3156ef27 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:21.112402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7525-f746-40e0-ba51-37eb3156ef27 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:21.113125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb7525-f746-40e0-ba51-37eb3156ef27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:21.115359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb7525-f746-40e0-ba51-37eb3156ef27 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:21.193146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 643/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Jun 15 14:39:20 2022] POST /identity/v3/services => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:39:23.346654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e9465-3c89-4ac5-9674-0031f8969595 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:23.347191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e9465-3c89-4ac5-9674-0031f8969595 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:23.347191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e9465-3c89-4ac5-9674-0031f8969595 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:23.348566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 653/1296] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:23.353689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eb429-7bc5-417d-85c2-2e336a34c7a4 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:23.354159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eb429-7bc5-417d-85c2-2e336a34c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:23.354274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eb429-7bc5-417d-85c2-2e336a34c7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:23.371115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-426eb429-7bc5-417d-85c2-2e336a34c7a4 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:23.372935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-426eb429-7bc5-417d-85c2-2e336a34c7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:23.447411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 644/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:23.471489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e9465-3c89-4ac5-9674-0031f8969595 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:23.576362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e9465-3c89-4ac5-9674-0031f8969595 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:23.611375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:23.612092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:23.612373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:23.612647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:23.613307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:23.614823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:23.627550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:23.628742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:23.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 654/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:39:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:23.634849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eb429-7bc5-417d-85c2-2e336a34c7a4 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:23.655444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:23.738909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426eb429-7bc5-417d-85c2-2e336a34c7a4 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:23.773851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:23.774538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:23.774820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:23.775095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:23.775801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] RBAC: Adding query filter params (name=placement) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:23.776108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:23.778452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:23.783831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 645/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Jun 15 14:39:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:23.789772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:23.823588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:23.896407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb7be7-e1d8-423c-a16c-c6cde4fe27e9 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:23.932361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f09c0-1a94-4f8c-9bd1-5f76875ad564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:23.933017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f09c0-1a94-4f8c-9bd1-5f76875ad564 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:23.933295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f09c0-1a94-4f8c-9bd1-5f76875ad564 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:23.933565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f09c0-1a94-4f8c-9bd1-5f76875ad564 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:23.935124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f09c0-1a94-4f8c-9bd1-5f76875ad564 None admin] RBAC: Adding query filter params (interface=public, service_id=d410a86c12674664a2119a6caa72b88b, region_id=RegionOne) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:23.935438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f09c0-1a94-4f8c-9bd1-5f76875ad564 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:23.936828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f09c0-1a94-4f8c-9bd1-5f76875ad564 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:23.942074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 655/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Jun 15 14:39:23 2022] GET /identity/v3/endpoints?service_id=d410a86c12674664a2119a6caa72b88b&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:23.948106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:24.034159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:24.052250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d49a6-05ee-4751-8a81-25c3efd05a9c None admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:24.087097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36deb6-2cba-42c0-b7ba-57881e953a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:24.088024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36deb6-2cba-42c0-b7ba-57881e953a13 None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:24.088305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36deb6-2cba-42c0-b7ba-57881e953a13 None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:24.088577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36deb6-2cba-42c0-b7ba-57881e953a13 None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:24.089250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36deb6-2cba-42c0-b7ba-57881e953a13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:24.091296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36deb6-2cba-42c0-b7ba-57881e953a13 None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:24.097946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 646/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Jun 15 14:39:23 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:39:26.298976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af270b-a4a4-4179-9c18-8d47fb3c2fa9 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:26.367962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af270b-a4a4-4179-9c18-8d47fb3c2fa9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:26.367962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af270b-a4a4-4179-9c18-8d47fb3c2fa9 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:26.367962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 656/1302] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:39:26.368300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db1690-0c53-4f02-96b2-4e6dba92cf50 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:26.368300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db1690-0c53-4f02-96b2-4e6dba92cf50 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:26.368300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db1690-0c53-4f02-96b2-4e6dba92cf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:26.368300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-45db1690-0c53-4f02-96b2-4e6dba92cf50 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:39:26.368300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-45db1690-0c53-4f02-96b2-4e6dba92cf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:39:26.397010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 647/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:39:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 14:39:26.421580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af270b-a4a4-4179-9c18-8d47fb3c2fa9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:26.522813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:26.530438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af270b-a4a4-4179-9c18-8d47fb3c2fa9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:26.566700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:26.567419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:26.567700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:26.567981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:26.568693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:26.570077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 14:39:26.582469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 14:39:26.588587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application with Lock( Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self._enter() Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application return self.creator() Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application created_value = creator( Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: ERROR keystone.server.flask.application Jun 15 14:39:26.589637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 657/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Jun 15 14:39:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 14:39:26.591485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db1690-0c53-4f02-96b2-4e6dba92cf50 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:26.671890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:26.694518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db1690-0c53-4f02-96b2-4e6dba92cf50 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:26.728988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f667f03-3275-49bb-9360-1e503676edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:26.729651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f667f03-3275-49bb-9360-1e503676edad None admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:26.729939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f667f03-3275-49bb-9360-1e503676edad None admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:26.730239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f667f03-3275-49bb-9360-1e503676edad None admin] PATH_INFO: `/v3/services` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:26.731236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f667f03-3275-49bb-9360-1e503676edad None admin] RBAC: Adding query filter params (name=placement) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:39:26.731546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f667f03-3275-49bb-9360-1e503676edad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:26.732930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f667f03-3275-49bb-9360-1e503676edad None admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:26.738484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 648/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Jun 15 14:39:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:39:26.744572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:39:26.846658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bacb3a-130a-455a-af7d-390001af0ac5 None admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:39:26.880321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55459fb9-4d15-4b44-a6aa-6da738fcdf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:39:26.880985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55459fb9-4d15-4b44-a6aa-6da738fcdf6d None admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:39:26.881270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55459fb9-4d15-4b44-a6aa-6da738fcdf6d None admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:39:26.881544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55459fb9-4d15-4b44-a6aa-6da738fcdf6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:39:26.882310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55459fb9-4d15-4b44-a6aa-6da738fcdf6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:39:26.883709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55459fb9-4d15-4b44-a6aa-6da738fcdf6d None admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:39:26.964688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 658/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Jun 15 14:39:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:39:36.764233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=30 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:39:36.959432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:10.964259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939055c-5e2c-4a67-a498-5dd92443907b None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:10.964830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939055c-5e2c-4a67-a498-5dd92443907b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:10.964830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939055c-5e2c-4a67-a498-5dd92443907b None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:10.966234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 649/1307] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:47:10.971918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f4c06-37ba-4811-9ee8-9760b978e900 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:10.973135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f4c06-37ba-4811-9ee8-9760b978e900 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:10.973730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f4c06-37ba-4811-9ee8-9760b978e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:11.095469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-bd9f4c06-37ba-4811-9ee8-9760b978e900 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:11.096687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bd9f4c06-37ba-4811-9ee8-9760b978e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:11.231041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 659/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:11.297265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fc424-4971-4c93-b110-373698c34dbf None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:11.297553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fc424-4971-4c93-b110-373698c34dbf None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:11.297981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fc424-4971-4c93-b110-373698c34dbf None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:11.299334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 650/1309] 10.222.0.23 () {54 vars in 964 bytes} [Wed Jun 15 14:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:11.307018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62098db2-da0f-4f7d-977a-060e0da69195 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:11.307322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62098db2-da0f-4f7d-977a-060e0da69195 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:11.307598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62098db2-da0f-4f7d-977a-060e0da69195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:11.394435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-62098db2-da0f-4f7d-977a-060e0da69195 None None] MFA Rules not processed for user `bead604b09be482c906a97b53bb03d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:11.395699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-62098db2-da0f-4f7d-977a-060e0da69195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:11.421187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:11.496542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fc424-4971-4c93-b110-373698c34dbf None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:11.498538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9b2fc424-4971-4c93-b110-373698c34dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:11.503346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 660/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Jun 15 14:47:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:11.632819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fc424-4971-4c93-b110-373698c34dbf None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:11.715378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e79123-7bfb-4a67-bae4-e953973c5429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:11.716294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79123-7bfb-4a67-bae4-e953973c5429 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:11.716746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79123-7bfb-4a67-bae4-e953973c5429 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:11.717194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e79123-7bfb-4a67-bae4-e953973c5429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:11.719009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-03e79123-7bfb-4a67-bae4-e953973c5429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:11.775930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:11.834642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e79123-7bfb-4a67-bae4-e953973c5429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:11.836084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e79123-7bfb-4a67-bae4-e953973c5429 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:11.856315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 651/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:47:14.449559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bacf3-787c-4497-8ece-d652c2dd14cc None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.450128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bacf3-787c-4497-8ece-d652c2dd14cc None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.450128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bacf3-787c-4497-8ece-d652c2dd14cc None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.451545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 661/1312] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:14.456591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a275a6e-1844-4bd9-b890-b090ce89e327 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.457014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a275a6e-1844-4bd9-b890-b090ce89e327 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.457156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a275a6e-1844-4bd9-b890-b090ce89e327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.535675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1a275a6e-1844-4bd9-b890-b090ce89e327 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:14.536829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1a275a6e-1844-4bd9-b890-b090ce89e327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.666306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 652/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:14.717681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f803dd-d94c-4cb7-9644-64ef93d83129 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.717965 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f803dd-d94c-4cb7-9644-64ef93d83129 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.718272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f803dd-d94c-4cb7-9644-64ef93d83129 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.719652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 662/1314] 10.222.0.23 () {54 vars in 964 bytes} [Wed Jun 15 14:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:14.725494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69d688-a37d-4dbb-b4f9-4c712ec46f1e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:14.725782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69d688-a37d-4dbb-b4f9-4c712ec46f1e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:14.726061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69d688-a37d-4dbb-b4f9-4c712ec46f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:14.737055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:14.801780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-da69d688-a37d-4dbb-b4f9-4c712ec46f1e None None] MFA Rules not processed for user `bead604b09be482c906a97b53bb03d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:14.802947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-da69d688-a37d-4dbb-b4f9-4c712ec46f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.878955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 653/1315] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Jun 15 14:47:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:14.888267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f803dd-d94c-4cb7-9644-64ef93d83129 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:14.889795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-06f803dd-d94c-4cb7-9644-64ef93d83129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:14.996446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f803dd-d94c-4cb7-9644-64ef93d83129 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:15.065382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:15.067700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f208c-e4c4-488e-83a7-b8fa28489934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:15.068343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f208c-e4c4-488e-83a7-b8fa28489934 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:15.068621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f208c-e4c4-488e-83a7-b8fa28489934 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:15.068888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f208c-e4c4-488e-83a7-b8fa28489934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:15.070567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6e5f208c-e4c4-488e-83a7-b8fa28489934 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:15.177629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f208c-e4c4-488e-83a7-b8fa28489934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:15.179108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f208c-e4c4-488e-83a7-b8fa28489934 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:15.198138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 663/1316] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:18.085992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fec5f-f0cf-4d33-950e-92af5d895cc7 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:18.086596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fec5f-f0cf-4d33-950e-92af5d895cc7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:18.086596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fec5f-f0cf-4d33-950e-92af5d895cc7 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:18.088361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 654/1317] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:18.093650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f2b4-ff93-45e3-bd6e-340b748d7f74 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:18.094086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f2b4-ff93-45e3-bd6e-340b748d7f74 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:18.094226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f2b4-ff93-45e3-bd6e-340b748d7f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:18.159496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-5343f2b4-ff93-45e3-bd6e-340b748d7f74 None None] MFA Rules not processed for user `dfee2b3a74c5497aa967fe8ef757093d`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:18.160728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5343f2b4-ff93-45e3-bd6e-340b748d7f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:18.213942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:18.258645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 664/1318] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:18.298555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fec5f-f0cf-4d33-950e-92af5d895cc7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:18.300751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-977fec5f-f0cf-4d33-950e-92af5d895cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:18.407786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fec5f-f0cf-4d33-950e-92af5d895cc7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:18.410676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:18.483545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e31ae-fffd-4ab0-adda-aa66a4370174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:18.484197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e31ae-fffd-4ab0-adda-aa66a4370174 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:18.484474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e31ae-fffd-4ab0-adda-aa66a4370174 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:18.484744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e31ae-fffd-4ab0-adda-aa66a4370174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:18.486424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7b8e31ae-fffd-4ab0-adda-aa66a4370174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:18.599653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e31ae-fffd-4ab0-adda-aa66a4370174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:18.601109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e31ae-fffd-4ab0-adda-aa66a4370174 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:18.620958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 655/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:21.776113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5e3c6-7e0e-4271-bee1-532076b53aa6 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:21.776711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5e3c6-7e0e-4271-bee1-532076b53aa6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:21.776711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5e3c6-7e0e-4271-bee1-532076b53aa6 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:21.778058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 665/1320] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:21.783164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7df924-7df4-4761-b349-e6834c510be1 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:21.783547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7df924-7df4-4761-b349-e6834c510be1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:21.783652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7df924-7df4-4761-b349-e6834c510be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:21.825651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:21.849619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-9b7df924-7df4-4761-b349-e6834c510be1 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:21.850862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9b7df924-7df4-4761-b349-e6834c510be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:21.947689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 656/1321] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:21.987024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5e3c6-7e0e-4271-bee1-532076b53aa6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:21.988571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9db5e3c6-7e0e-4271-bee1-532076b53aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:22.091632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5e3c6-7e0e-4271-bee1-532076b53aa6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:22.165696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6814315-1316-4ad8-bb12-dd6e3e3d746b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:22.166379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6814315-1316-4ad8-bb12-dd6e3e3d746b service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:22.166663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6814315-1316-4ad8-bb12-dd6e3e3d746b service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:22.166937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6814315-1316-4ad8-bb12-dd6e3e3d746b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:22.168586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b6814315-1316-4ad8-bb12-dd6e3e3d746b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:22.175961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:22.276297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6814315-1316-4ad8-bb12-dd6e3e3d746b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:22.277758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6814315-1316-4ad8-bb12-dd6e3e3d746b service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:22.295647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 666/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:25.843864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4963547-c43e-4eaf-99bf-fa40a8ff0334 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:25.844408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4963547-c43e-4eaf-99bf-fa40a8ff0334 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:25.844408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4963547-c43e-4eaf-99bf-fa40a8ff0334 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:25.845752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 657/1323] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:25.851235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95e4d9-d6a5-4246-9acf-37d0f2d5a614 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:25.851810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95e4d9-d6a5-4246-9acf-37d0f2d5a614 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:25.851810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95e4d9-d6a5-4246-9acf-37d0f2d5a614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:25.934373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6d95e4d9-d6a5-4246-9acf-37d0f2d5a614 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:25.935930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6d95e4d9-d6a5-4246-9acf-37d0f2d5a614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:26.009921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:26.071891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 667/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:26.109415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4963547-c43e-4eaf-99bf-fa40a8ff0334 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:26.111137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c4963547-c43e-4eaf-99bf-fa40a8ff0334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:26.224507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4963547-c43e-4eaf-99bf-fa40a8ff0334 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:26.237670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:26.305198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701939f-7bdb-441a-862e-e09d9ecd02f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:26.306470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701939f-7bdb-441a-862e-e09d9ecd02f9 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:26.306786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701939f-7bdb-441a-862e-e09d9ecd02f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:26.307080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701939f-7bdb-441a-862e-e09d9ecd02f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:26.308854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b701939f-7bdb-441a-862e-e09d9ecd02f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:26.423931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701939f-7bdb-441a-862e-e09d9ecd02f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:26.425278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701939f-7bdb-441a-862e-e09d9ecd02f9 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:26.444607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 658/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:28.775169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524af5d-3d6d-44d5-ae55-54079306c421 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.775738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524af5d-3d6d-44d5-ae55-54079306c421 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.775738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524af5d-3d6d-44d5-ae55-54079306c421 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.777123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 668/1326] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:28.782555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834414c6-3aee-46eb-a3fd-3dcbff94bf03 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:28.783001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834414c6-3aee-46eb-a3fd-3dcbff94bf03 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:28.783091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834414c6-3aee-46eb-a3fd-3dcbff94bf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:28.835227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:28.856915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-834414c6-3aee-46eb-a3fd-3dcbff94bf03 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:28.858425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-834414c6-3aee-46eb-a3fd-3dcbff94bf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:28.967078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 659/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:29.007669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524af5d-3d6d-44d5-ae55-54079306c421 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:29.010270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f524af5d-3d6d-44d5-ae55-54079306c421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:29.120138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524af5d-3d6d-44d5-ae55-54079306c421 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:29.199705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f79840-4131-45e1-bb55-634f30912ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:29.200360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f79840-4131-45e1-bb55-634f30912ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:29.200639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f79840-4131-45e1-bb55-634f30912ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:29.200910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f79840-4131-45e1-bb55-634f30912ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:29.202632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-99f79840-4131-45e1-bb55-634f30912ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:29.220301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:29.328037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f79840-4131-45e1-bb55-634f30912ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:29.329842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f79840-4131-45e1-bb55-634f30912ae7 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:29.359745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 669/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:31.950681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100eb098-d259-4141-be72-03e6abae23aa None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:31.951213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100eb098-d259-4141-be72-03e6abae23aa None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:31.951213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100eb098-d259-4141-be72-03e6abae23aa None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:31.952611 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 660/1329] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:31.958007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90a5b1-1f37-4e0c-8d80-29c9e82a3798 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:31.958486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90a5b1-1f37-4e0c-8d80-29c9e82a3798 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:31.958582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90a5b1-1f37-4e0c-8d80-29c9e82a3798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:32.029183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1e90a5b1-1f37-4e0c-8d80-29c9e82a3798 None None] MFA Rules not processed for user `dfee2b3a74c5497aa967fe8ef757093d`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:32.030474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1e90a5b1-1f37-4e0c-8d80-29c9e82a3798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:32.097312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:32.130740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 670/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:32.171626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100eb098-d259-4141-be72-03e6abae23aa None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:32.173179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-100eb098-d259-4141-be72-03e6abae23aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:32.316804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100eb098-d259-4141-be72-03e6abae23aa None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:32.344406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:32.405453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80217b2-89c8-4732-9058-51489e62934e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:32.406136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80217b2-89c8-4732-9058-51489e62934e service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:32.406447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80217b2-89c8-4732-9058-51489e62934e service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:32.406718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80217b2-89c8-4732-9058-51489e62934e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:32.408383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a80217b2-89c8-4732-9058-51489e62934e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:32.541694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80217b2-89c8-4732-9058-51489e62934e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:32.543222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80217b2-89c8-4732-9058-51489e62934e service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:32.565473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 661/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:35.259034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa661b-d0a7-4bf3-b716-49b97211fae8 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:35.259601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa661b-d0a7-4bf3-b716-49b97211fae8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:35.259601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa661b-d0a7-4bf3-b716-49b97211fae8 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:35.260958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 671/1332] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:35.266137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c0ca-42b8-4cfa-9e05-8f2a11f30012 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:35.266547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c0ca-42b8-4cfa-9e05-8f2a11f30012 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:35.266718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c0ca-42b8-4cfa-9e05-8f2a11f30012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:35.329117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:35.332015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5076c0ca-42b8-4cfa-9e05-8f2a11f30012 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:35.333219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5076c0ca-42b8-4cfa-9e05-8f2a11f30012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:35.429070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 662/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:35.465159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa661b-d0a7-4bf3-b716-49b97211fae8 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:35.466796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cdaa661b-d0a7-4bf3-b716-49b97211fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:35.571829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa661b-d0a7-4bf3-b716-49b97211fae8 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:35.651471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282db7de-c789-49a4-b358-f93dff2dcbd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:35.652121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282db7de-c789-49a4-b358-f93dff2dcbd9 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:35.652397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282db7de-c789-49a4-b358-f93dff2dcbd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:35.652668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282db7de-c789-49a4-b358-f93dff2dcbd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:35.654338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-282db7de-c789-49a4-b358-f93dff2dcbd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:35.681209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:35.763342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282db7de-c789-49a4-b358-f93dff2dcbd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:35.764788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282db7de-c789-49a4-b358-f93dff2dcbd9 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:35.782247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 672/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:37.993943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed670f-0703-46d4-a276-308082f7058b None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:37.994553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed670f-0703-46d4-a276-308082f7058b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:37.994553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed670f-0703-46d4-a276-308082f7058b None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:37.995972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 663/1335] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:38.001299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38831ec8-5e38-43ea-9042-a394c39e1cc3 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:38.001750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38831ec8-5e38-43ea-9042-a394c39e1cc3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:38.001845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38831ec8-5e38-43ea-9042-a394c39e1cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:38.069415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-38831ec8-5e38-43ea-9042-a394c39e1cc3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:38.070711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-38831ec8-5e38-43ea-9042-a394c39e1cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:38.148409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:38.170800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 673/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:38.211540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed670f-0703-46d4-a276-308082f7058b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:38.213214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-82ed670f-0703-46d4-a276-308082f7058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:38.319365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed670f-0703-46d4-a276-308082f7058b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:38.351044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:38.400610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bf9f4-109a-4c35-8a31-c9141e99668e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:38.401268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bf9f4-109a-4c35-8a31-c9141e99668e service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:38.401551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bf9f4-109a-4c35-8a31-c9141e99668e service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:38.401827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bf9f4-109a-4c35-8a31-c9141e99668e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:38.403607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-353bf9f4-109a-4c35-8a31-c9141e99668e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:38.518972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bf9f4-109a-4c35-8a31-c9141e99668e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:38.520414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bf9f4-109a-4c35-8a31-c9141e99668e service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:38.544147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 664/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:41.281628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d689d0a-592e-4476-8e75-d0621e132e17 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:41.282347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d689d0a-592e-4476-8e75-d0621e132e17 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:41.282347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d689d0a-592e-4476-8e75-d0621e132e17 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:41.283723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 674/1338] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:41.289731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2c1f7-05c1-4a05-af98-b6bb8b9fe368 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:41.293953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2c1f7-05c1-4a05-af98-b6bb8b9fe368 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:41.293953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2c1f7-05c1-4a05-af98-b6bb8b9fe368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:41.360208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-63a2c1f7-05c1-4a05-af98-b6bb8b9fe368 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:41.361465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-63a2c1f7-05c1-4a05-af98-b6bb8b9fe368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:41.380160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:41.470957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 665/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:41.508685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d689d0a-592e-4476-8e75-d0621e132e17 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:41.513105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9d689d0a-592e-4476-8e75-d0621e132e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:41.632150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d689d0a-592e-4476-8e75-d0621e132e17 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:41.636351 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f964ec-bfc3-418d-8baf-c0c353eb04a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:41.637059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f964ec-bfc3-418d-8baf-c0c353eb04a4 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:41.637340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f964ec-bfc3-418d-8baf-c0c353eb04a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:41.637613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f964ec-bfc3-418d-8baf-c0c353eb04a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:41.639478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-52f964ec-bfc3-418d-8baf-c0c353eb04a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:41.757187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f964ec-bfc3-418d-8baf-c0c353eb04a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:41.758700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f964ec-bfc3-418d-8baf-c0c353eb04a4 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:41.769731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:47:41.780567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 675/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:46.559261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b20234-71ed-4bb2-a890-987fea67d774 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:46.560072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b20234-71ed-4bb2-a890-987fea67d774 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:46.560072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b20234-71ed-4bb2-a890-987fea67d774 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:46.562454 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 666/1341] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:46.566819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e6f7-a809-4572-8346-ed5604c2c70a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:46.567338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e6f7-a809-4572-8346-ed5604c2c70a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:46.567441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e6f7-a809-4572-8346-ed5604c2c70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:46.626690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f841e6f7-a809-4572-8346-ed5604c2c70a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:46.628260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f841e6f7-a809-4572-8346-ed5604c2c70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:46.738201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 676/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:46.778424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b20234-71ed-4bb2-a890-987fea67d774 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:46.791787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b20234-71ed-4bb2-a890-987fea67d774 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:46.795513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8e825-2cb5-4b94-92ea-b505a0f0cd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:46.796156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e825-2cb5-4b94-92ea-b505a0f0cd31 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:46.796460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e825-2cb5-4b94-92ea-b505a0f0cd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:46.796731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e825-2cb5-4b94-92ea-b505a0f0cd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:46.806958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8e825-2cb5-4b94-92ea-b505a0f0cd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:46.808539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8e825-2cb5-4b94-92ea-b505a0f0cd31 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:46.818982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 667/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:50.151017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf621dcd-1686-4ba8-b146-0a2b98153b8f None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:50.151551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf621dcd-1686-4ba8-b146-0a2b98153b8f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:50.152227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf621dcd-1686-4ba8-b146-0a2b98153b8f None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:50.153422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 677/1344] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:50.160079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74decd09-9c0f-4795-9b17-2c0a5dde481d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:50.160604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74decd09-9c0f-4795-9b17-2c0a5dde481d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:50.161115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74decd09-9c0f-4795-9b17-2c0a5dde481d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:50.196538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-74decd09-9c0f-4795-9b17-2c0a5dde481d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:50.197866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-74decd09-9c0f-4795-9b17-2c0a5dde481d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:50.205628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 668/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:50.242962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf621dcd-1686-4ba8-b146-0a2b98153b8f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:50.253489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf621dcd-1686-4ba8-b146-0a2b98153b8f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:50.257117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af257dcf-f3be-467d-88d7-88899cbb7a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:50.257751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af257dcf-f3be-467d-88d7-88899cbb7a15 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:50.258036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af257dcf-f3be-467d-88d7-88899cbb7a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:50.258346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af257dcf-f3be-467d-88d7-88899cbb7a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:50.267910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af257dcf-f3be-467d-88d7-88899cbb7a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:50.269376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af257dcf-f3be-467d-88d7-88899cbb7a15 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:50.279970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 678/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:54.683787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b33f49-2695-456c-a624-969f88f57c7b None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:54.684284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b33f49-2695-456c-a624-969f88f57c7b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:54.684379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b33f49-2695-456c-a624-969f88f57c7b None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:54.685791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 669/1347] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:54.691528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abcd4f-1982-4f36-85b9-ba9c8daaf2de None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:54.692038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abcd4f-1982-4f36-85b9-ba9c8daaf2de None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:54.692245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abcd4f-1982-4f36-85b9-ba9c8daaf2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:54.711841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f3abcd4f-1982-4f36-85b9-ba9c8daaf2de None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:47:54.713071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f3abcd4f-1982-4f36-85b9-ba9c8daaf2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:47:54.722108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 679/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:47:54.761558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b33f49-2695-456c-a624-969f88f57c7b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:47:54.772580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b33f49-2695-456c-a624-969f88f57c7b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:47:54.776828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9dc3e-7fe7-4c04-942f-6beeb6d9da7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:47:54.777489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9dc3e-7fe7-4c04-942f-6beeb6d9da7b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:54.777767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9dc3e-7fe7-4c04-942f-6beeb6d9da7b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:54.778041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9dc3e-7fe7-4c04-942f-6beeb6d9da7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:54.788122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9dc3e-7fe7-4c04-942f-6beeb6d9da7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:47:54.789672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9dc3e-7fe7-4c04-942f-6beeb6d9da7b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:47:54.799690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 670/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:47:59.938979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f1745-b263-4d9b-b967-fd21f85df48d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:59.939543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f1745-b263-4d9b-b967-fd21f85df48d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:59.945292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f1745-b263-4d9b-b967-fd21f85df48d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:47:59.947646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 680/1350] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:47:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:47:59.966138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8992d0-475c-49c4-bc96-443244e7777a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:47:59.967567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8992d0-475c-49c4-bc96-443244e7777a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:47:59.968010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8992d0-475c-49c4-bc96-443244e7777a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:00.070062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-bb8992d0-475c-49c4-bc96-443244e7777a None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:00.071211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bb8992d0-475c-49c4-bc96-443244e7777a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:00.122710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:00.155428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 671/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:04.447591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04d5bb-75ab-4a3e-b17b-6c3b33d0053e None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:04.447591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04d5bb-75ab-4a3e-b17b-6c3b33d0053e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:04.447591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04d5bb-75ab-4a3e-b17b-6c3b33d0053e None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:04.449116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 681/1352] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:04.454699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98359f-43e1-4a2c-a164-6b059b1ecb2e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:04.455550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98359f-43e1-4a2c-a164-6b059b1ecb2e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:04.456144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98359f-43e1-4a2c-a164-6b059b1ecb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:04.475988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7e98359f-43e1-4a2c-a164-6b059b1ecb2e None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:04.477179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7e98359f-43e1-4a2c-a164-6b059b1ecb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:04.487066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 672/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:04.705933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=56 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:06.048286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c21d5-d7b5-427c-a774-f7dc181516d4 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:06.048921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c21d5-d7b5-427c-a774-f7dc181516d4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:06.048921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c21d5-d7b5-427c-a774-f7dc181516d4 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:06.050230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 682/1354] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:06.055223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2690-0ac8-4655-b881-a0a171ec0fdc None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:06.055609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2690-0ac8-4655-b881-a0a171ec0fdc None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:06.055985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2690-0ac8-4655-b881-a0a171ec0fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:06.074245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-81df2690-0ac8-4655-b881-a0a171ec0fdc None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:06.075456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-81df2690-0ac8-4655-b881-a0a171ec0fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:06.084266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 673/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:09.965194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577570d-c694-473a-b2a8-928ef0cf9d3c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:09.965769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577570d-c694-473a-b2a8-928ef0cf9d3c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:09.965769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577570d-c694-473a-b2a8-928ef0cf9d3c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:09.967653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 683/1356] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:09.972558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f551-9d46-4fcf-9045-33630155fa19 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:09.972971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f551-9d46-4fcf-9045-33630155fa19 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:09.973139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f551-9d46-4fcf-9045-33630155fa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:09.991181 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2c41f551-9d46-4fcf-9045-33630155fa19 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:09.992368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2c41f551-9d46-4fcf-9045-33630155fa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:10.000720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 674/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:10.053799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936c524-fcc0-475e-935f-a97dafb82507 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:10.054592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936c524-fcc0-475e-935f-a97dafb82507 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:10.055183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936c524-fcc0-475e-935f-a97dafb82507 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:10.057388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 684/1358] 10.222.0.23 () {54 vars in 961 bytes} [Wed Jun 15 14:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:10.064383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662a99-86ec-422b-bdbf-44c453fb6aed None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:10.065064 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662a99-86ec-422b-bdbf-44c453fb6aed None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:10.065863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662a99-86ec-422b-bdbf-44c453fb6aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:10.120399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-60662a99-86ec-422b-bdbf-44c453fb6aed None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:10.121939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-60662a99-86ec-422b-bdbf-44c453fb6aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:10.216753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 675/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Jun 15 14:48:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:10.225998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936c524-fcc0-475e-935f-a97dafb82507 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:10.242146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936c524-fcc0-475e-935f-a97dafb82507 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:10.248931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d9922-3f7c-4064-a17c-39164908771b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:10.249724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d9922-3f7c-4064-a17c-39164908771b service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:10.250080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d9922-3f7c-4064-a17c-39164908771b service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:10.250494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d9922-3f7c-4064-a17c-39164908771b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:10.260921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d9922-3f7c-4064-a17c-39164908771b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:10.262685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d9922-3f7c-4064-a17c-39164908771b service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:10.273067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 685/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:12.685876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafe188-1bf3-4bcf-bd66-ca102d7aff3f None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:12.686533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafe188-1bf3-4bcf-bd66-ca102d7aff3f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:12.686533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafe188-1bf3-4bcf-bd66-ca102d7aff3f None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:12.687813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 676/1361] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:12.693336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152408-fcb2-4956-a142-b93bcf211ba8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:12.693794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152408-fcb2-4956-a142-b93bcf211ba8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:12.693918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152408-fcb2-4956-a142-b93bcf211ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:12.715908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-68152408-fcb2-4956-a142-b93bcf211ba8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:12.717653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-68152408-fcb2-4956-a142-b93bcf211ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:12.733794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 686/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:12.773773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafe188-1bf3-4bcf-bd66-ca102d7aff3f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:12.792037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafe188-1bf3-4bcf-bd66-ca102d7aff3f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:12.797982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09735bcd-4f8d-41d5-b06d-0c8de00d40d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:12.799244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735bcd-4f8d-41d5-b06d-0c8de00d40d2 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:12.799710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735bcd-4f8d-41d5-b06d-0c8de00d40d2 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:12.800147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735bcd-4f8d-41d5-b06d-0c8de00d40d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:12.810732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09735bcd-4f8d-41d5-b06d-0c8de00d40d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:12.812643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09735bcd-4f8d-41d5-b06d-0c8de00d40d2 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:12.825689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 677/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:15.482484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b61f4-0aa3-4888-8a20-b3bb50c84f50 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:15.483028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b61f4-0aa3-4888-8a20-b3bb50c84f50 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:15.483144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b61f4-0aa3-4888-8a20-b3bb50c84f50 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:15.484487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 687/1364] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:15.489951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1320-2a11-407f-ad41-dd3be178d787 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:15.490413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1320-2a11-407f-ad41-dd3be178d787 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:15.490554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1320-2a11-407f-ad41-dd3be178d787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:15.510651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c82f1320-2a11-407f-ad41-dd3be178d787 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:15.511774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c82f1320-2a11-407f-ad41-dd3be178d787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:15.519723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 678/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:15.567365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c386-face-431d-b669-d7433dd5a109 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:15.567795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c386-face-431d-b669-d7433dd5a109 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:15.568172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c386-face-431d-b669-d7433dd5a109 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:15.569675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 688/1366] 10.222.0.23 () {54 vars in 961 bytes} [Wed Jun 15 14:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:15.575847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9716b4-8a5d-4dea-8ae2-740c8e866a33 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:15.576141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9716b4-8a5d-4dea-8ae2-740c8e866a33 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:15.576424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9716b4-8a5d-4dea-8ae2-740c8e866a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:15.594607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2a9716b4-8a5d-4dea-8ae2-740c8e866a33 None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:15.595799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2a9716b4-8a5d-4dea-8ae2-740c8e866a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:15.605505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 679/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Jun 15 14:48:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:15.615583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c386-face-431d-b669-d7433dd5a109 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:15.630620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c386-face-431d-b669-d7433dd5a109 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:15.636918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75ca7b-fa14-4508-b8ce-bcccad78444c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:15.637557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75ca7b-fa14-4508-b8ce-bcccad78444c service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:15.637842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75ca7b-fa14-4508-b8ce-bcccad78444c service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:15.638117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75ca7b-fa14-4508-b8ce-bcccad78444c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:15.646367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75ca7b-fa14-4508-b8ce-bcccad78444c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:15.647967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75ca7b-fa14-4508-b8ce-bcccad78444c service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:15.657677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 689/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:18.078685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b61a07-7d64-4e63-85e4-f5d78ea993da None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.079229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b61a07-7d64-4e63-85e4-f5d78ea993da None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.079229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b61a07-7d64-4e63-85e4-f5d78ea993da None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.080606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 680/1369] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:18.085854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b131d2-f8eb-4ad0-85bf-958cc29f49b8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.086702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b131d2-f8eb-4ad0-85bf-958cc29f49b8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.086975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b131d2-f8eb-4ad0-85bf-958cc29f49b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.106089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-39b131d2-f8eb-4ad0-85bf-958cc29f49b8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:18.107378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-39b131d2-f8eb-4ad0-85bf-958cc29f49b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:18.119961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 690/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:18.161134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b61a07-7d64-4e63-85e4-f5d78ea993da None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:18.177018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b61a07-7d64-4e63-85e4-f5d78ea993da None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:18.177631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf52fc-271d-4653-a9d4-06c75532a5a6 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.178100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf52fc-271d-4653-a9d4-06c75532a5a6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.178576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf52fc-271d-4653-a9d4-06c75532a5a6 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.180095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 691/1371] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:48:18.185179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897f4ac-8118-4db1-b45f-ede889c083e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:18.185835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897f4ac-8118-4db1-b45f-ede889c083e0 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.186120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897f4ac-8118-4db1-b45f-ede889c083e0 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.186440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897f4ac-8118-4db1-b45f-ede889c083e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.186729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c5bef-b017-46e0-b967-c70e081d0b3c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.187333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c5bef-b017-46e0-b967-c70e081d0b3c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.187837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c5bef-b017-46e0-b967-c70e081d0b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.195331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897f4ac-8118-4db1-b45f-ede889c083e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:18.197079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897f4ac-8118-4db1-b45f-ede889c083e0 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:18.207097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 681/1372] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:18.208869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c24c5bef-b017-46e0-b967-c70e081d0b3c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:18.210147 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c24c5bef-b017-46e0-b967-c70e081d0b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:18.219267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 692/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:18.534999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e75e-1dc2-4173-93b1-c9911d3d5df5 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.534999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e75e-1dc2-4173-93b1-c9911d3d5df5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.534999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e75e-1dc2-4173-93b1-c9911d3d5df5 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.539882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 682/1374] 10.222.0.23 () {54 vars in 963 bytes} [Wed Jun 15 14:48:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:18.546385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be631-e051-461f-abc4-0b4224aa9944 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.546718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be631-e051-461f-abc4-0b4224aa9944 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.546958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176be631-e051-461f-abc4-0b4224aa9944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.604808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-176be631-e051-461f-abc4-0b4224aa9944 None None] MFA Rules not processed for user `8d29198c768d4024af04273a3cca8f20`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:18.606205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-176be631-e051-461f-abc4-0b4224aa9944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:18.677262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 693/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Wed Jun 15 14:48:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:18.685432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7e75e-1dc2-4173-93b1-c9911d3d5df5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:18.697810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7e75e-1dc2-4173-93b1-c9911d3d5df5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:18.702507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03273ba-b1a6-40c8-8839-256a9d3fbd6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:18.703219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03273ba-b1a6-40c8-8839-256a9d3fbd6d service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:18.703505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03273ba-b1a6-40c8-8839-256a9d3fbd6d service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:18.703782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03273ba-b1a6-40c8-8839-256a9d3fbd6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:18.717003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03273ba-b1a6-40c8-8839-256a9d3fbd6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:18.717003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03273ba-b1a6-40c8-8839-256a9d3fbd6d service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:18.726451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 683/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:20.600395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc960411-b999-421c-99f7-713fc66a9926 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:20.600917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc960411-b999-421c-99f7-713fc66a9926 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:20.601043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc960411-b999-421c-99f7-713fc66a9926 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:20.602414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 694/1377] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:20.608313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2597d-8013-4c62-a4bb-2bd8df3d7d38 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:20.610260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2597d-8013-4c62-a4bb-2bd8df3d7d38 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:20.610260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2597d-8013-4c62-a4bb-2bd8df3d7d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:20.629193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-77b2597d-8013-4c62-a4bb-2bd8df3d7d38 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:20.630470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-77b2597d-8013-4c62-a4bb-2bd8df3d7d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:20.638979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 684/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:20.685266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc960411-b999-421c-99f7-713fc66a9926 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:20.700704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc960411-b999-421c-99f7-713fc66a9926 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:20.706335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712269e-faf9-4f48-a4bb-5779a2fb67fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:20.706985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712269e-faf9-4f48-a4bb-5779a2fb67fc service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:20.707275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712269e-faf9-4f48-a4bb-5779a2fb67fc service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:20.707546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712269e-faf9-4f48-a4bb-5779a2fb67fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:20.717239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712269e-faf9-4f48-a4bb-5779a2fb67fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:20.719010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712269e-faf9-4f48-a4bb-5779a2fb67fc service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:20.728695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 695/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:23.079950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b85b57-e6fb-4541-bce5-4e7fc28aa202 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:23.080473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b85b57-e6fb-4541-bce5-4e7fc28aa202 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:23.080473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b85b57-e6fb-4541-bce5-4e7fc28aa202 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:23.081835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 685/1380] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:23.087127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f68d2-8b47-4dd5-a3c2-7b5a07a9fddc None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:23.087573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f68d2-8b47-4dd5-a3c2-7b5a07a9fddc None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:23.087727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f68d2-8b47-4dd5-a3c2-7b5a07a9fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:23.105856 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-690f68d2-8b47-4dd5-a3c2-7b5a07a9fddc None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:23.107085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-690f68d2-8b47-4dd5-a3c2-7b5a07a9fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:23.114983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 696/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:23.159594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b85b57-e6fb-4541-bce5-4e7fc28aa202 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:23.175429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b85b57-e6fb-4541-bce5-4e7fc28aa202 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:23.181711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d519b-dd72-47ff-a503-426da83c80b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:23.182530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d519b-dd72-47ff-a503-426da83c80b8 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:23.182890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d519b-dd72-47ff-a503-426da83c80b8 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:23.183285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d519b-dd72-47ff-a503-426da83c80b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:23.191872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d519b-dd72-47ff-a503-426da83c80b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:23.193537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d519b-dd72-47ff-a503-426da83c80b8 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:23.197983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:23.204474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 686/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:25.576837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230d9f-9899-4131-ba76-4ef73ec8eaa7 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:25.577384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230d9f-9899-4131-ba76-4ef73ec8eaa7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:25.577384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230d9f-9899-4131-ba76-4ef73ec8eaa7 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:25.578928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 697/1383] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:25.584615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4296a9-43ae-4e31-bd4a-3b6b65140ddf None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:25.585423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4296a9-43ae-4e31-bd4a-3b6b65140ddf None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:25.585770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4296a9-43ae-4e31-bd4a-3b6b65140ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:25.605637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4a4296a9-43ae-4e31-bd4a-3b6b65140ddf None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:25.606949 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4a4296a9-43ae-4e31-bd4a-3b6b65140ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:25.622257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 687/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:25.658820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37230d9f-9899-4131-ba76-4ef73ec8eaa7 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:25.674809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37230d9f-9899-4131-ba76-4ef73ec8eaa7 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:25.680488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f1f23-a5f2-4ae6-923a-d95da30fad84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:25.681133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f1f23-a5f2-4ae6-923a-d95da30fad84 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:25.682048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f1f23-a5f2-4ae6-923a-d95da30fad84 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:25.682721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f1f23-a5f2-4ae6-923a-d95da30fad84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:25.691886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f1f23-a5f2-4ae6-923a-d95da30fad84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:25.693595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f1f23-a5f2-4ae6-923a-d95da30fad84 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:25.704260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 698/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:27.918418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5f65-33f1-4231-82fa-dd5b559a253d None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:27.918952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5f65-33f1-4231-82fa-dd5b559a253d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:27.918952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5f65-33f1-4231-82fa-dd5b559a253d None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:27.920457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 688/1386] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:27.925666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8d98e-326f-43e4-b5b1-00ab9fb380b7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:27.926080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8d98e-326f-43e4-b5b1-00ab9fb380b7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:27.926262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8d98e-326f-43e4-b5b1-00ab9fb380b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:27.943801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-66e8d98e-326f-43e4-b5b1-00ab9fb380b7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:27.945007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-66e8d98e-326f-43e4-b5b1-00ab9fb380b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:27.953543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 699/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:27.997824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5f65-33f1-4231-82fa-dd5b559a253d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:28.013131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5f65-33f1-4231-82fa-dd5b559a253d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:28.019043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d309575-f352-4f13-bba6-fb83b9ba75c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:28.019694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d309575-f352-4f13-bba6-fb83b9ba75c0 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:28.019989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d309575-f352-4f13-bba6-fb83b9ba75c0 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:28.020380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d309575-f352-4f13-bba6-fb83b9ba75c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:28.029200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d309575-f352-4f13-bba6-fb83b9ba75c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:28.030832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d309575-f352-4f13-bba6-fb83b9ba75c0 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:28.045619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 689/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:30.439424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce1d3-86b7-4feb-a922-ec1a28499c96 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:30.439958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce1d3-86b7-4feb-a922-ec1a28499c96 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:30.440235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce1d3-86b7-4feb-a922-ec1a28499c96 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:30.442067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 700/1389] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:30.447951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d11621-f2a3-46a9-b5e9-b1835ed7ba31 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:30.448517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d11621-f2a3-46a9-b5e9-b1835ed7ba31 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:30.448517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d11621-f2a3-46a9-b5e9-b1835ed7ba31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:30.467865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-24d11621-f2a3-46a9-b5e9-b1835ed7ba31 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:30.469734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-24d11621-f2a3-46a9-b5e9-b1835ed7ba31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:30.482822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 690/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:30.531578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce1d3-86b7-4feb-a922-ec1a28499c96 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:30.546062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce1d3-86b7-4feb-a922-ec1a28499c96 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:30.551556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126de550-199f-4c0e-87a2-9dee583c06f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:30.552209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126de550-199f-4c0e-87a2-9dee583c06f2 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:30.552497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126de550-199f-4c0e-87a2-9dee583c06f2 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:30.552773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126de550-199f-4c0e-87a2-9dee583c06f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:30.562518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126de550-199f-4c0e-87a2-9dee583c06f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:30.564126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126de550-199f-4c0e-87a2-9dee583c06f2 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:30.574595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 701/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:33.066807 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71edcf-cfb3-486d-b9f1-c71c67fb02f7 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:33.067360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71edcf-cfb3-486d-b9f1-c71c67fb02f7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:33.067360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71edcf-cfb3-486d-b9f1-c71c67fb02f7 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:33.068729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 691/1392] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:33.074524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36f92-423d-429b-bcf1-79e157a082a8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:33.074959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36f92-423d-429b-bcf1-79e157a082a8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:33.075109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36f92-423d-429b-bcf1-79e157a082a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:33.094469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6cf36f92-423d-429b-bcf1-79e157a082a8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:33.095675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6cf36f92-423d-429b-bcf1-79e157a082a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:33.103552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 702/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:33.147877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71edcf-cfb3-486d-b9f1-c71c67fb02f7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:33.162817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71edcf-cfb3-486d-b9f1-c71c67fb02f7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:33.168429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc461c-ac19-4d71-a654-68ded5eff87f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:33.169065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc461c-ac19-4d71-a654-68ded5eff87f service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:33.169352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc461c-ac19-4d71-a654-68ded5eff87f service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:33.169633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc461c-ac19-4d71-a654-68ded5eff87f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:33.177719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc461c-ac19-4d71-a654-68ded5eff87f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:33.179337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc461c-ac19-4d71-a654-68ded5eff87f service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:33.188750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 692/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:34.491136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6b6d-544c-4b7e-9bf4-89ffc0e493fb None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:34.491679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6b6d-544c-4b7e-9bf4-89ffc0e493fb None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:34.491679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6b6d-544c-4b7e-9bf4-89ffc0e493fb None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:34.493141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 703/1395] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:34.499084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323d786-07c0-424f-8f0c-b374b65bf238 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:34.499507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323d786-07c0-424f-8f0c-b374b65bf238 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:34.499638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323d786-07c0-424f-8f0c-b374b65bf238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:34.518654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6323d786-07c0-424f-8f0c-b374b65bf238 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:34.519899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6323d786-07c0-424f-8f0c-b374b65bf238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:34.529618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 693/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:34.586317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfc3c1-4e6a-4c93-a601-0ae6ab995d8d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:34.586608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfc3c1-4e6a-4c93-a601-0ae6ab995d8d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:34.586888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfc3c1-4e6a-4c93-a601-0ae6ab995d8d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:34.588945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 704/1397] 10.222.0.23 () {54 vars in 962 bytes} [Wed Jun 15 14:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:34.594204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1456-144b-4613-82ea-1b5326e4835f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:34.594455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1456-144b-4613-82ea-1b5326e4835f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:34.594772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1456-144b-4613-82ea-1b5326e4835f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:34.645206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ffea1456-144b-4613-82ea-1b5326e4835f None None] MFA Rules not processed for user `28faa0c6ac9b4e3a9528c3199506257c`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:34.646455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ffea1456-144b-4613-82ea-1b5326e4835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:34.716731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 694/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Wed Jun 15 14:48:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:34.724313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfc3c1-4e6a-4c93-a601-0ae6ab995d8d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:34.735083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfc3c1-4e6a-4c93-a601-0ae6ab995d8d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:34.738801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2aadb-3168-47f9-93b4-7b834c58dfd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:34.739439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2aadb-3168-47f9-93b4-7b834c58dfd0 service glance] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:34.739726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2aadb-3168-47f9-93b4-7b834c58dfd0 service glance] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:34.740000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2aadb-3168-47f9-93b4-7b834c58dfd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:34.746411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:34.750001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2aadb-3168-47f9-93b4-7b834c58dfd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:34.751656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2aadb-3168-47f9-93b4-7b834c58dfd0 service glance] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:34.761516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 705/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:35.570845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba154c-51ec-4f36-9bc6-5241a0a9e2d3 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:35.571375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba154c-51ec-4f36-9bc6-5241a0a9e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:35.571375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba154c-51ec-4f36-9bc6-5241a0a9e2d3 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:35.573079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 695/1400] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:35.578292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f0213-aa46-4720-a252-611363d45dbb None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:35.578707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f0213-aa46-4720-a252-611363d45dbb None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:35.578841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f0213-aa46-4720-a252-611363d45dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:35.596777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7a6f0213-aa46-4720-a252-611363d45dbb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:35.597966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7a6f0213-aa46-4720-a252-611363d45dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:35.606926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 706/1401] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:35.652674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba154c-51ec-4f36-9bc6-5241a0a9e2d3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:35.662979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb058c18-a881-48f5-895d-5b2a9bd5a86f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:35.667416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb058c18-a881-48f5-895d-5b2a9bd5a86f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:35.667416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb058c18-a881-48f5-895d-5b2a9bd5a86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:35.669284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba154c-51ec-4f36-9bc6-5241a0a9e2d3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:35.674754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8823c-9791-4e35-91ac-dbe2a07b3bdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:35.675540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8823c-9791-4e35-91ac-dbe2a07b3bdb service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:35.675822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8823c-9791-4e35-91ac-dbe2a07b3bdb service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:35.676095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8823c-9791-4e35-91ac-dbe2a07b3bdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:35.684863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8823c-9791-4e35-91ac-dbe2a07b3bdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:35.686657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8823c-9791-4e35-91ac-dbe2a07b3bdb service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:35.696336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 696/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:35.713370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-bb058c18-a881-48f5-895d-5b2a9bd5a86f None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:35.714716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bb058c18-a881-48f5-895d-5b2a9bd5a86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:35.777641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 707/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:35.824007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be10bf-a472-4de6-9c2b-6c0fc88ca704 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:35.824316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be10bf-a472-4de6-9c2b-6c0fc88ca704 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:35.824727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be10bf-a472-4de6-9c2b-6c0fc88ca704 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:35.825963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 697/1404] 10.222.0.23 () {54 vars in 942 bytes} [Wed Jun 15 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:35.840076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deb21b-5080-4cca-809d-307c2708c725 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:35.840076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deb21b-5080-4cca-809d-307c2708c725 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:35.840076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37deb21b-5080-4cca-809d-307c2708c725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:35.884607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-37deb21b-5080-4cca-809d-307c2708c725 None None] MFA Rules not processed for user `0b3a2fb853764be199cc6ff0dfd18064`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:35.885958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-37deb21b-5080-4cca-809d-307c2708c725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:35.964949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 708/1405] 10.222.0.23 () {58 vars in 1039 bytes} [Wed Jun 15 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:48:35.974587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be10bf-a472-4de6-9c2b-6c0fc88ca704 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:35.989682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be10bf-a472-4de6-9c2b-6c0fc88ca704 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:35.995698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c835ac7-f2f5-4cec-b97c-6ade5ceda84c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:35.996375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c835ac7-f2f5-4cec-b97c-6ade5ceda84c service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:35.996655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c835ac7-f2f5-4cec-b97c-6ade5ceda84c service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:35.996930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c835ac7-f2f5-4cec-b97c-6ade5ceda84c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:36.006290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c835ac7-f2f5-4cec-b97c-6ade5ceda84c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:36.007962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c835ac7-f2f5-4cec-b97c-6ade5ceda84c service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:36.017868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 698/1406] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:48:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:48:38.085499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90df2-39e6-4984-ba29-a93a0228e425 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:38.086120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90df2-39e6-4984-ba29-a93a0228e425 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:38.086120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90df2-39e6-4984-ba29-a93a0228e425 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:38.087465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 709/1407] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:48:38.093510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b121-7ce1-4c93-8f98-e9229bbd0e4a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:38.093958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b121-7ce1-4c93-8f98-e9229bbd0e4a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:38.094062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b121-7ce1-4c93-8f98-e9229bbd0e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:38.114603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-af62b121-7ce1-4c93-8f98-e9229bbd0e4a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:48:38.115796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-af62b121-7ce1-4c93-8f98-e9229bbd0e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:48:38.127328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 699/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:48:38.172326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90df2-39e6-4984-ba29-a93a0228e425 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:48:38.188249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90df2-39e6-4984-ba29-a93a0228e425 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:48:38.199431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64c936-758b-4fbb-ae14-7592e43fac54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:48:38.200101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64c936-758b-4fbb-ae14-7592e43fac54 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:48:38.200384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64c936-758b-4fbb-ae14-7592e43fac54 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:48:38.200660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64c936-758b-4fbb-ae14-7592e43fac54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:48:38.209580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64c936-758b-4fbb-ae14-7592e43fac54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:48:38.211228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64c936-758b-4fbb-ae14-7592e43fac54 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:48:38.221776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 710/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:48:48.110224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=75 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:48:48.217255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:00.187234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b3892-de99-49fc-8def-e8069e00c370 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:00.187860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b3892-de99-49fc-8def-e8069e00c370 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:00.187860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b3892-de99-49fc-8def-e8069e00c370 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:00.189154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 700/1410] 10.222.0.23 () {54 vars in 918 bytes} [Wed Jun 15 14:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:00.194571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee399f-476f-4cd3-83af-c03d3a9db130 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:00.195271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee399f-476f-4cd3-83af-c03d3a9db130 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:00.195271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee399f-476f-4cd3-83af-c03d3a9db130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:00.212849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-32ee399f-476f-4cd3-83af-c03d3a9db130 None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:00.214156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-32ee399f-476f-4cd3-83af-c03d3a9db130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:00.222776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 711/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Jun 15 14:49:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:49:00.238029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20299996-359d-4306-a48f-ea7a5c48c808 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:00.238419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20299996-359d-4306-a48f-ea7a5c48c808 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:00.238773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20299996-359d-4306-a48f-ea7a5c48c808 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:00.240122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 701/1412] 10.222.0.23 () {54 vars in 918 bytes} [Wed Jun 15 14:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:00.245529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166513b3-4b95-4ae1-9abb-dc8bef6f038c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:00.245529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166513b3-4b95-4ae1-9abb-dc8bef6f038c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:00.246202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166513b3-4b95-4ae1-9abb-dc8bef6f038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:00.264553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-166513b3-4b95-4ae1-9abb-dc8bef6f038c None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:00.265677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-166513b3-4b95-4ae1-9abb-dc8bef6f038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:00.274980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 712/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Jun 15 14:49:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:02.893853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d300d-457d-4740-b858-fc22fa4ee339 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:02.894414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d300d-457d-4740-b858-fc22fa4ee339 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:02.894414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d300d-457d-4740-b858-fc22fa4ee339 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:02.895708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:02.900899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9cbe8-8d45-4468-883c-7abfe9721950 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:02.901301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9cbe8-8d45-4468-883c-7abfe9721950 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:02.901523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9cbe8-8d45-4468-883c-7abfe9721950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:02.919920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-efb9cbe8-8d45-4468-883c-7abfe9721950 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:02.921100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-efb9cbe8-8d45-4468-883c-7abfe9721950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:02.928698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:02.972017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d300d-457d-4740-b858-fc22fa4ee339 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:02.986379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d300d-457d-4740-b858-fc22fa4ee339 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:02.992079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832b397-dfed-4260-9679-5985188d5624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:02.992733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b397-dfed-4260-9679-5985188d5624 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:02.993014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b397-dfed-4260-9679-5985188d5624 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:02.993284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b397-dfed-4260-9679-5985188d5624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:03.004476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832b397-dfed-4260-9679-5985188d5624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:03.006991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832b397-dfed-4260-9679-5985188d5624 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:03.018555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 703/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:03.965848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda477-34b7-415a-b2f9-8bea83f8c86e None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:03.966444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda477-34b7-415a-b2f9-8bea83f8c86e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:03.966557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda477-34b7-415a-b2f9-8bea83f8c86e None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:03.967842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:03.972827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cdf91-c96e-402f-ad23-674813459eb3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:03.973260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cdf91-c96e-402f-ad23-674813459eb3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:03.973364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cdf91-c96e-402f-ad23-674813459eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:03.991370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c08cdf91-c96e-402f-ad23-674813459eb3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:03.992556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c08cdf91-c96e-402f-ad23-674813459eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:04.000889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:04.163547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadda477-34b7-415a-b2f9-8bea83f8c86e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:04.178513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadda477-34b7-415a-b2f9-8bea83f8c86e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:04.186852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aed96-e948-495c-a12f-66bf3cd1dc2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:04.187498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aed96-e948-495c-a12f-66bf3cd1dc2e service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:04.187776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aed96-e948-495c-a12f-66bf3cd1dc2e service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:04.188048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aed96-e948-495c-a12f-66bf3cd1dc2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:04.197430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aed96-e948-495c-a12f-66bf3cd1dc2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:04.200366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aed96-e948-495c-a12f-66bf3cd1dc2e service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:04.210088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 715/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:05.134279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80206f-689b-4d4a-88d8-18fec0c6e617 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:05.134839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80206f-689b-4d4a-88d8-18fec0c6e617 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:05.134839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80206f-689b-4d4a-88d8-18fec0c6e617 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:05.136812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:05.141945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f637cc-58ea-4005-a163-95586033f5f4 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:05.142496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f637cc-58ea-4005-a163-95586033f5f4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:05.142796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f637cc-58ea-4005-a163-95586033f5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:05.160725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f4f637cc-58ea-4005-a163-95586033f5f4 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:05.161904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f4f637cc-58ea-4005-a163-95586033f5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:05.170510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:05.327732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80206f-689b-4d4a-88d8-18fec0c6e617 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:05.342099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80206f-689b-4d4a-88d8-18fec0c6e617 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:05.347507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb708-e879-4ade-bdaf-4c0e98aabfd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:05.348140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb708-e879-4ade-bdaf-4c0e98aabfd3 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:05.348424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb708-e879-4ade-bdaf-4c0e98aabfd3 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:05.348701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb708-e879-4ade-bdaf-4c0e98aabfd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:05.357753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb708-e879-4ade-bdaf-4c0e98aabfd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:05.359378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88deb708-e879-4ade-bdaf-4c0e98aabfd3 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:05.368877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 706/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:06.291930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816a771-9089-4e88-9200-0e4b6b2409ed None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:06.292492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816a771-9089-4e88-9200-0e4b6b2409ed None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:06.292604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816a771-9089-4e88-9200-0e4b6b2409ed None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:06.293946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:06.298778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0267c3-10e9-4915-9a8a-83fd2a0ac21c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:06.299198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0267c3-10e9-4915-9a8a-83fd2a0ac21c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:06.299349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0267c3-10e9-4915-9a8a-83fd2a0ac21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:06.317257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-bc0267c3-10e9-4915-9a8a-83fd2a0ac21c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:06.318486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bc0267c3-10e9-4915-9a8a-83fd2a0ac21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:06.326846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:06.478505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816a771-9089-4e88-9200-0e4b6b2409ed None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:06.493069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816a771-9089-4e88-9200-0e4b6b2409ed None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:06.499582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5894f7-da50-4501-924f-54f1b90377e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:06.500061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5894f7-da50-4501-924f-54f1b90377e7 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:06.500345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5894f7-da50-4501-924f-54f1b90377e7 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:06.500620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5894f7-da50-4501-924f-54f1b90377e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:06.508745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5894f7-da50-4501-924f-54f1b90377e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:06.510384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5894f7-da50-4501-924f-54f1b90377e7 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:06.520362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 718/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:07.443375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb38374-641a-46ab-a0ce-8017bcba3959 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:07.443962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb38374-641a-46ab-a0ce-8017bcba3959 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:07.443962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb38374-641a-46ab-a0ce-8017bcba3959 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:07.445333 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:07.450763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f782a9-aca6-4ec1-a22a-4fd7bde5de8c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:07.451180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f782a9-aca6-4ec1-a22a-4fd7bde5de8c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:07.451342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f782a9-aca6-4ec1-a22a-4fd7bde5de8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:07.469602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e9f782a9-aca6-4ec1-a22a-4fd7bde5de8c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:07.470822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e9f782a9-aca6-4ec1-a22a-4fd7bde5de8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:07.478496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:07.631567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb38374-641a-46ab-a0ce-8017bcba3959 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:07.646155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb38374-641a-46ab-a0ce-8017bcba3959 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:07.651632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a6c7c-0cdf-444e-9733-974612fb44db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:07.652265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a6c7c-0cdf-444e-9733-974612fb44db service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:07.652548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a6c7c-0cdf-444e-9733-974612fb44db service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:07.652821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a6c7c-0cdf-444e-9733-974612fb44db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:07.660867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a6c7c-0cdf-444e-9733-974612fb44db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:07.662891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a6c7c-0cdf-444e-9733-974612fb44db service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:07.672825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 709/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:08.594884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b26f2-6aee-41fe-af51-a836db475b32 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:08.595443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b26f2-6aee-41fe-af51-a836db475b32 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:08.595443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b26f2-6aee-41fe-af51-a836db475b32 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:08.596836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:08.601713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898e51a-39d7-4548-8699-0b427f9b8ac6 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:08.602147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898e51a-39d7-4548-8699-0b427f9b8ac6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:08.602298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898e51a-39d7-4548-8699-0b427f9b8ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:08.620334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3898e51a-39d7-4548-8699-0b427f9b8ac6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:08.621486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3898e51a-39d7-4548-8699-0b427f9b8ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:08.629646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:08.783705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b26f2-6aee-41fe-af51-a836db475b32 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:08.798152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b26f2-6aee-41fe-af51-a836db475b32 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:08.803565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bdc9f-8d81-457e-80af-fbd7d9bc314f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:08.804194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bdc9f-8d81-457e-80af-fbd7d9bc314f service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:08.804473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bdc9f-8d81-457e-80af-fbd7d9bc314f service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:08.804744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bdc9f-8d81-457e-80af-fbd7d9bc314f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:08.813822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bdc9f-8d81-457e-80af-fbd7d9bc314f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:08.815438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bdc9f-8d81-457e-80af-fbd7d9bc314f service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:08.826095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 721/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:09.750848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bddc5-1fc5-470e-b412-f72f58664a0c None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:09.751427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bddc5-1fc5-470e-b412-f72f58664a0c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:09.751427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bddc5-1fc5-470e-b412-f72f58664a0c None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:09.752822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:09.757728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9507f-c879-490c-9ea1-7c9cd8c51b15 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:09.758112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9507f-c879-490c-9ea1-7c9cd8c51b15 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:09.758307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9507f-c879-490c-9ea1-7c9cd8c51b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:09.776914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-5ce9507f-c879-490c-9ea1-7c9cd8c51b15 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:09.778093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5ce9507f-c879-490c-9ea1-7c9cd8c51b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:09.786150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:09.936753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bddc5-1fc5-470e-b412-f72f58664a0c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:09.951222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bddc5-1fc5-470e-b412-f72f58664a0c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:09.956494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477afe38-96cf-499e-83ff-bef063102d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:09.957125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477afe38-96cf-499e-83ff-bef063102d2b service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:09.957407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477afe38-96cf-499e-83ff-bef063102d2b service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:09.957682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477afe38-96cf-499e-83ff-bef063102d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:09.966261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477afe38-96cf-499e-83ff-bef063102d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:09.967853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477afe38-96cf-499e-83ff-bef063102d2b service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:09.977104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 712/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:10.896734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859540e-43f9-4741-aabe-9167e1f5fed3 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:10.897587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859540e-43f9-4741-aabe-9167e1f5fed3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:10.897587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859540e-43f9-4741-aabe-9167e1f5fed3 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:10.898780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:10.905763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95fbed-3349-480b-ad62-04b4fc6d5b38 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:10.906266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95fbed-3349-480b-ad62-04b4fc6d5b38 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:10.906383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95fbed-3349-480b-ad62-04b4fc6d5b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:10.924517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ba95fbed-3349-480b-ad62-04b4fc6d5b38 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:10.925698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ba95fbed-3349-480b-ad62-04b4fc6d5b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:10.934509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:11.088971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859540e-43f9-4741-aabe-9167e1f5fed3 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:11.103573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859540e-43f9-4741-aabe-9167e1f5fed3 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:11.109331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab57c8-1dc7-476b-9987-1dc80375097d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:11.109961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab57c8-1dc7-476b-9987-1dc80375097d service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:11.110276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab57c8-1dc7-476b-9987-1dc80375097d service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:11.110551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab57c8-1dc7-476b-9987-1dc80375097d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:11.118611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab57c8-1dc7-476b-9987-1dc80375097d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:11.121256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab57c8-1dc7-476b-9987-1dc80375097d service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:11.130687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 724/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:12.084739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc55f18-a173-47a6-bb49-96a3685f4d37 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:12.085310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc55f18-a173-47a6-bb49-96a3685f4d37 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:12.085310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc55f18-a173-47a6-bb49-96a3685f4d37 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:12.086745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:12.092127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cef9d-9cf5-4042-b038-b336b30045da None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:12.092591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cef9d-9cf5-4042-b038-b336b30045da None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:12.092713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cef9d-9cf5-4042-b038-b336b30045da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:12.111297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a19cef9d-9cf5-4042-b038-b336b30045da None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:12.112529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a19cef9d-9cf5-4042-b038-b336b30045da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:12.121056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:12.270902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc55f18-a173-47a6-bb49-96a3685f4d37 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:12.287543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc55f18-a173-47a6-bb49-96a3685f4d37 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:12.293306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941f296-1995-4a9a-ac4e-22098f0d556e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:12.293957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f296-1995-4a9a-ac4e-22098f0d556e service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:12.294285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f296-1995-4a9a-ac4e-22098f0d556e service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:12.294558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f296-1995-4a9a-ac4e-22098f0d556e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:12.303528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941f296-1995-4a9a-ac4e-22098f0d556e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:12.305113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941f296-1995-4a9a-ac4e-22098f0d556e service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:12.314546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 715/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:14.360641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b63101-4eb7-4db6-8ad2-642e0b8206e1 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:14.361304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b63101-4eb7-4db6-8ad2-642e0b8206e1 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:14.361304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b63101-4eb7-4db6-8ad2-642e0b8206e1 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:14.362590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:14.367832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714e6a-3731-43b5-9723-8742d6d636fb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:14.368227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714e6a-3731-43b5-9723-8742d6d636fb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:14.368375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714e6a-3731-43b5-9723-8742d6d636fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:14.386652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-03714e6a-3731-43b5-9723-8742d6d636fb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:14.387850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-03714e6a-3731-43b5-9723-8742d6d636fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:14.395622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:14.434092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b63101-4eb7-4db6-8ad2-642e0b8206e1 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:14.444808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b63101-4eb7-4db6-8ad2-642e0b8206e1 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:14.448977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426ac97-5f2d-4173-9b28-0ae99b874887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:14.449992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426ac97-5f2d-4173-9b28-0ae99b874887 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:14.450307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426ac97-5f2d-4173-9b28-0ae99b874887 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:14.450582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426ac97-5f2d-4173-9b28-0ae99b874887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:14.459331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426ac97-5f2d-4173-9b28-0ae99b874887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:14.460770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426ac97-5f2d-4173-9b28-0ae99b874887 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:14.470876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 727/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:16.804140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2f257-4671-4ff3-a22a-8d9a4d16bcab None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:16.804731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2f257-4671-4ff3-a22a-8d9a4d16bcab None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:16.804731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2f257-4671-4ff3-a22a-8d9a4d16bcab None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:16.806101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:16.812008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f1f34-0f55-44ec-8ff9-5aea5fb06438 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:16.812425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f1f34-0f55-44ec-8ff9-5aea5fb06438 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:16.812557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f1f34-0f55-44ec-8ff9-5aea5fb06438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:16.830765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-419f1f34-0f55-44ec-8ff9-5aea5fb06438 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:16.831940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-419f1f34-0f55-44ec-8ff9-5aea5fb06438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:16.840621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:16.880224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2f257-4671-4ff3-a22a-8d9a4d16bcab None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:16.891398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2f257-4671-4ff3-a22a-8d9a4d16bcab None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:16.894881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b9e33-ba15-472a-895c-60117387d4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:16.895512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b9e33-ba15-472a-895c-60117387d4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:16.895796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b9e33-ba15-472a-895c-60117387d4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:16.896075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b9e33-ba15-472a-895c-60117387d4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:16.906020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b9e33-ba15-472a-895c-60117387d4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:16.908304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b9e33-ba15-472a-895c-60117387d4f4 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:16.919120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 718/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:19.538683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59bad-333c-4348-b28c-deaf208ead40 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:19.539142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59bad-333c-4348-b28c-deaf208ead40 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:19.539250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59bad-333c-4348-b28c-deaf208ead40 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:19.540622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:19.546433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb30f4-2860-4bfe-8ad2-439c31c04c0a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:19.546832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb30f4-2860-4bfe-8ad2-439c31c04c0a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:19.546995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb30f4-2860-4bfe-8ad2-439c31c04c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:19.565233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c3eb30f4-2860-4bfe-8ad2-439c31c04c0a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:19.566491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c3eb30f4-2860-4bfe-8ad2-439c31c04c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:19.574483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:19.613096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59bad-333c-4348-b28c-deaf208ead40 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:19.624709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59bad-333c-4348-b28c-deaf208ead40 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:19.628285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9bdd-1908-4045-a548-9e0da157dfa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:19.628923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9bdd-1908-4045-a548-9e0da157dfa5 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:19.629207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9bdd-1908-4045-a548-9e0da157dfa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:19.629482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9bdd-1908-4045-a548-9e0da157dfa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:19.639084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9bdd-1908-4045-a548-9e0da157dfa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:19.640524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9bdd-1908-4045-a548-9e0da157dfa5 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:19.650014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 730/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:21.960755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da1acf-c523-4521-8aeb-df021b8114e0 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:21.961313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da1acf-c523-4521-8aeb-df021b8114e0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:21.961313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da1acf-c523-4521-8aeb-df021b8114e0 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:21.962706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:21.968826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c13d1-6446-4181-afd5-112227ea7dd3 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:21.969276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c13d1-6446-4181-afd5-112227ea7dd3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:21.969378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c13d1-6446-4181-afd5-112227ea7dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:21.986894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f63c13d1-6446-4181-afd5-112227ea7dd3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:21.988081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f63c13d1-6446-4181-afd5-112227ea7dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:21.996161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:22.036649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da1acf-c523-4521-8aeb-df021b8114e0 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:22.038390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-45da1acf-c523-4521-8aeb-df021b8114e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:22.051726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da1acf-c523-4521-8aeb-df021b8114e0 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:22.055281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f98c1-9bf1-4d12-8d20-98c207054f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:22.055942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f98c1-9bf1-4d12-8d20-98c207054f3e service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:22.056229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f98c1-9bf1-4d12-8d20-98c207054f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:22.056512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f98c1-9bf1-4d12-8d20-98c207054f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:22.066621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f98c1-9bf1-4d12-8d20-98c207054f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:22.068085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f98c1-9bf1-4d12-8d20-98c207054f3e service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:22.079163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 721/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:24.350001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33058458-bc8f-4cb0-804e-c72dbe01f115 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:24.350589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33058458-bc8f-4cb0-804e-c72dbe01f115 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:24.350589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33058458-bc8f-4cb0-804e-c72dbe01f115 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:24.351939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:24.357688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b790fbd-5a65-428b-b8d4-867bdb8a4fd8 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:24.358137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b790fbd-5a65-428b-b8d4-867bdb8a4fd8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:24.358260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b790fbd-5a65-428b-b8d4-867bdb8a4fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:24.376156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2b790fbd-5a65-428b-b8d4-867bdb8a4fd8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:24.377410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2b790fbd-5a65-428b-b8d4-867bdb8a4fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:24.385431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:24.427563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33058458-bc8f-4cb0-804e-c72dbe01f115 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:24.438722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33058458-bc8f-4cb0-804e-c72dbe01f115 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:24.442135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6356f-760e-43db-be97-b3774e6211c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:24.442791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6356f-760e-43db-be97-b3774e6211c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:24.443069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6356f-760e-43db-be97-b3774e6211c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:24.443339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6356f-760e-43db-be97-b3774e6211c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:24.453096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6356f-760e-43db-be97-b3774e6211c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:24.454568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6356f-760e-43db-be97-b3774e6211c1 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:24.463711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 733/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:26.711982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ec9f-5d4a-4635-810a-1374a3ccee1f None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:26.712552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ec9f-5d4a-4635-810a-1374a3ccee1f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:26.712552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ec9f-5d4a-4635-810a-1374a3ccee1f None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:26.713951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:26.719529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b6d3-2a1a-468c-9da9-dca93bac3b18 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:26.719957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b6d3-2a1a-468c-9da9-dca93bac3b18 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:26.720059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b6d3-2a1a-468c-9da9-dca93bac3b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:26.737652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7eb9b6d3-2a1a-468c-9da9-dca93bac3b18 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:26.738858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7eb9b6d3-2a1a-468c-9da9-dca93bac3b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:26.747483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:26.786733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175ec9f-5d4a-4635-810a-1374a3ccee1f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:26.798332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175ec9f-5d4a-4635-810a-1374a3ccee1f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:26.802331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adb07f-5559-49b8-ada9-941d26987519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:26.802984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adb07f-5559-49b8-ada9-941d26987519 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:26.803269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adb07f-5559-49b8-ada9-941d26987519 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:26.803546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adb07f-5559-49b8-ada9-941d26987519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:26.815366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adb07f-5559-49b8-ada9-941d26987519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:26.816822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adb07f-5559-49b8-ada9-941d26987519 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:26.827583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 724/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:29.053411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a374b7-8842-47df-8fb6-19d745bf5292 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:29.053977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a374b7-8842-47df-8fb6-19d745bf5292 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:29.053977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a374b7-8842-47df-8fb6-19d745bf5292 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:29.055387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:29.061437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3ff9e-097e-48e1-b570-0af942885faf None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:29.061880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3ff9e-097e-48e1-b570-0af942885faf None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:29.061977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3ff9e-097e-48e1-b570-0af942885faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:29.068078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=70 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:29.081314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-acd3ff9e-097e-48e1-b570-0af942885faf None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:29.084822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-acd3ff9e-097e-48e1-b570-0af942885faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:29.090631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:29.128621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a374b7-8842-47df-8fb6-19d745bf5292 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:29.134017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=73 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:29.140784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a374b7-8842-47df-8fb6-19d745bf5292 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:29.144260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc11c-df0f-46c8-8052-7c428b201bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:29.144897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc11c-df0f-46c8-8052-7c428b201bea service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:29.145179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc11c-df0f-46c8-8052-7c428b201bea service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:29.145460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc11c-df0f-46c8-8052-7c428b201bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:29.155440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc11c-df0f-46c8-8052-7c428b201bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:29.156892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc11c-df0f-46c8-8052-7c428b201bea service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:29.166349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 736/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:31.580425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e9283-d218-413b-9c0d-80fb1fe409b4 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:31.592624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e9283-d218-413b-9c0d-80fb1fe409b4 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:31.592624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e9283-d218-413b-9c0d-80fb1fe409b4 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:31.592624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:31.593352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95286383-63d6-443a-9694-bd21d904c92c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:31.593352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95286383-63d6-443a-9694-bd21d904c92c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:31.593352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95286383-63d6-443a-9694-bd21d904c92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:31.610197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-95286383-63d6-443a-9694-bd21d904c92c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:31.610197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-95286383-63d6-443a-9694-bd21d904c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:31.622960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:31.668803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e9283-d218-413b-9c0d-80fb1fe409b4 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:31.683698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e9283-d218-413b-9c0d-80fb1fe409b4 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:31.687432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aeb0b-48af-4283-92d3-a15217b64e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:31.688080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aeb0b-48af-4283-92d3-a15217b64e40 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:31.688418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aeb0b-48af-4283-92d3-a15217b64e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:31.688780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aeb0b-48af-4283-92d3-a15217b64e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:31.698041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aeb0b-48af-4283-92d3-a15217b64e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:31.700035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aeb0b-48af-4283-92d3-a15217b64e40 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:31.709438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 727/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:33.906565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b658bda-7b1d-4f32-9918-92b6ce65d405 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:33.907091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b658bda-7b1d-4f32-9918-92b6ce65d405 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:33.907091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b658bda-7b1d-4f32-9918-92b6ce65d405 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:33.908414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:33.913584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db198531-ae4c-4e67-8e79-550d8db1fd9d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:33.913966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db198531-ae4c-4e67-8e79-550d8db1fd9d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:33.914149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db198531-ae4c-4e67-8e79-550d8db1fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:33.931564 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-db198531-ae4c-4e67-8e79-550d8db1fd9d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:33.932756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-db198531-ae4c-4e67-8e79-550d8db1fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:33.940839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:33.978775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b658bda-7b1d-4f32-9918-92b6ce65d405 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:33.995288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b658bda-7b1d-4f32-9918-92b6ce65d405 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:34.000172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5468918-8f58-4559-9118-9b8cd2776662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:34.000818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468918-8f58-4559-9118-9b8cd2776662 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:34.001104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468918-8f58-4559-9118-9b8cd2776662 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:34.001449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468918-8f58-4559-9118-9b8cd2776662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:34.010734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5468918-8f58-4559-9118-9b8cd2776662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:34.012674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5468918-8f58-4559-9118-9b8cd2776662 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:34.022217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 739/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:36.162493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e182-d492-4154-8be5-22775aba7619 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:36.163065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e182-d492-4154-8be5-22775aba7619 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:36.163065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89e182-d492-4154-8be5-22775aba7619 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:36.164415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:36.170763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aec5b-2648-4d29-aeea-3053db25a734 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:36.171192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aec5b-2648-4d29-aeea-3053db25a734 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:36.171295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aec5b-2648-4d29-aeea-3053db25a734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:36.189370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ad6aec5b-2648-4d29-aeea-3053db25a734 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:36.190581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ad6aec5b-2648-4d29-aeea-3053db25a734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:36.198357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:36.235349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89e182-d492-4154-8be5-22775aba7619 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:36.246909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89e182-d492-4154-8be5-22775aba7619 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:36.250653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92d4dd-2589-443d-ac45-f19da05ec405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:36.251288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92d4dd-2589-443d-ac45-f19da05ec405 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:36.251568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92d4dd-2589-443d-ac45-f19da05ec405 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:36.251844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92d4dd-2589-443d-ac45-f19da05ec405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:36.260734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92d4dd-2589-443d-ac45-f19da05ec405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:36.262575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92d4dd-2589-443d-ac45-f19da05ec405 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:36.271786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 730/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:46.184492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=17 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:46.267652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=17 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:49:56.689219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890117f-5bc0-4186-b1ea-a8c64b02a826 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:56.689923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890117f-5bc0-4186-b1ea-a8c64b02a826 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:56.689923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890117f-5bc0-4186-b1ea-a8c64b02a826 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:56.691210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 741/1471] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:56.701193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3b417-9f7d-4387-930f-aa6f6579aabc None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:56.701648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3b417-9f7d-4387-930f-aa6f6579aabc None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:56.701752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3b417-9f7d-4387-930f-aa6f6579aabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:56.719135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b1a3b417-9f7d-4387-930f-aa6f6579aabc None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:56.720316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b1a3b417-9f7d-4387-930f-aa6f6579aabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:56.728054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 731/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:56.785833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac896-7b8c-432c-8bf5-f8e064b0a37a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:56.786122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac896-7b8c-432c-8bf5-f8e064b0a37a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:56.786435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac896-7b8c-432c-8bf5-f8e064b0a37a None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:56.787791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 742/1473] 10.222.0.23 () {54 vars in 962 bytes} [Wed Jun 15 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:56.793292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f998d-2c7f-40eb-b165-2ade9799871d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:56.793601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f998d-2c7f-40eb-b165-2ade9799871d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:56.793877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f998d-2c7f-40eb-b165-2ade9799871d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:56.812465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a88f998d-2c7f-40eb-b165-2ade9799871d None None] MFA Rules not processed for user `28faa0c6ac9b4e3a9528c3199506257c`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:56.813646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a88f998d-2c7f-40eb-b165-2ade9799871d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:56.819925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 732/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Wed Jun 15 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:49:56.828532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ac896-7b8c-432c-8bf5-f8e064b0a37a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:56.839086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ac896-7b8c-432c-8bf5-f8e064b0a37a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:56.842813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d6d6e-add4-400a-ba7e-b357a4c019b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:56.843442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6d6e-add4-400a-ba7e-b357a4c019b4 service glance] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:56.843726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6d6e-add4-400a-ba7e-b357a4c019b4 service glance] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:56.844002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6d6e-add4-400a-ba7e-b357a4c019b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:56.852692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d6d6e-add4-400a-ba7e-b357a4c019b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:56.854131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d6d6e-add4-400a-ba7e-b357a4c019b4 service glance] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:56.863768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 743/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:49:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:49:59.073865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425752f-7278-4be8-9ef2-1cbfa2196c8b None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.075298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425752f-7278-4be8-9ef2-1cbfa2196c8b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.075573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425752f-7278-4be8-9ef2-1cbfa2196c8b None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.076942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 733/1476] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:49:59.082153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe664f-caab-4248-a760-070a1d473e68 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.082615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe664f-caab-4248-a760-070a1d473e68 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.082735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe664f-caab-4248-a760-070a1d473e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.100999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c8fe664f-caab-4248-a760-070a1d473e68 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:49:59.102501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c8fe664f-caab-4248-a760-070a1d473e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:49:59.110729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 744/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:49:59.133518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425752f-7278-4be8-9ef2-1cbfa2196c8b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:49:59.145957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425752f-7278-4be8-9ef2-1cbfa2196c8b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:49:59.150830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174864f0-3b1e-42e7-a639-0d7f3aaf739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:49:59.151483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174864f0-3b1e-42e7-a639-0d7f3aaf739d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:49:59.151771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174864f0-3b1e-42e7-a639-0d7f3aaf739d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:49:59.152723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174864f0-3b1e-42e7-a639-0d7f3aaf739d admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:49:59.153562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174864f0-3b1e-42e7-a639-0d7f3aaf739d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:49:59.155561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-174864f0-3b1e-42e7-a639-0d7f3aaf739d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:49:59.155561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:49:59.155561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:49:59.156766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174864f0-3b1e-42e7-a639-0d7f3aaf739d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:49:59.176931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 734/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Jun 15 14:49:59 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:50:01.332717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf59eb-e974-4dfa-b01f-7da6d094a295 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.333302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf59eb-e974-4dfa-b01f-7da6d094a295 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.333302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf59eb-e974-4dfa-b01f-7da6d094a295 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.334729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 745/1479] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:01.340654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a214d6-e492-4396-b877-df8b13ac635c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.341068 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a214d6-e492-4396-b877-df8b13ac635c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.341210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a214d6-e492-4396-b877-df8b13ac635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.359475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c3a214d6-e492-4396-b877-df8b13ac635c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:01.360686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c3a214d6-e492-4396-b877-df8b13ac635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:01.368609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 735/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:01.411231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf59eb-e974-4dfa-b01f-7da6d094a295 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:01.425826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf59eb-e974-4dfa-b01f-7da6d094a295 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:01.431170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae7b07-d2e0-4267-8d17-58845fc35f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:01.433749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae7b07-d2e0-4267-8d17-58845fc35f12 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:01.433749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae7b07-d2e0-4267-8d17-58845fc35f12 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:01.433749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae7b07-d2e0-4267-8d17-58845fc35f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:01.440543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae7b07-d2e0-4267-8d17-58845fc35f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:01.442155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae7b07-d2e0-4267-8d17-58845fc35f12 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:01.451443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 746/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:03.805806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67305-bb8b-4cb8-a2f7-02d245f32b42 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.806377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67305-bb8b-4cb8-a2f7-02d245f32b42 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.806377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67305-bb8b-4cb8-a2f7-02d245f32b42 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.807683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 736/1482] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:03.813104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008fc0-15eb-4c95-9e69-d32268298a15 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.813543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008fc0-15eb-4c95-9e69-d32268298a15 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.813683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008fc0-15eb-4c95-9e69-d32268298a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.831812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-04008fc0-15eb-4c95-9e69-d32268298a15 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:03.833015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-04008fc0-15eb-4c95-9e69-d32268298a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:03.841458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 747/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:03.877098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e67305-bb8b-4cb8-a2f7-02d245f32b42 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:03.888368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e67305-bb8b-4cb8-a2f7-02d245f32b42 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:03.891824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51f457-61b9-418a-80bc-26f5fb4c95e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:03.892456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51f457-61b9-418a-80bc-26f5fb4c95e7 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:03.892734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51f457-61b9-418a-80bc-26f5fb4c95e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:03.893004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51f457-61b9-418a-80bc-26f5fb4c95e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:03.901717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51f457-61b9-418a-80bc-26f5fb4c95e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:03.903839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51f457-61b9-418a-80bc-26f5fb4c95e7 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:03.912964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 737/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:06.151173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a8cd6-8762-462c-9662-77059443be04 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.151717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a8cd6-8762-462c-9662-77059443be04 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.151717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a8cd6-8762-462c-9662-77059443be04 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.154404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 748/1485] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:50:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:06.159565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097dd63-ea82-4508-be67-15791187cfc5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.160027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097dd63-ea82-4508-be67-15791187cfc5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.160123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097dd63-ea82-4508-be67-15791187cfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.177955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-d097dd63-ea82-4508-be67-15791187cfc5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:06.179269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d097dd63-ea82-4508-be67-15791187cfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:06.187161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 738/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:50:06.225800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a8cd6-8762-462c-9662-77059443be04 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:06.236531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a8cd6-8762-462c-9662-77059443be04 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:06.240025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fa5b0-a700-4aa6-a534-c466438dd2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:06.240658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fa5b0-a700-4aa6-a534-c466438dd2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:06.240940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fa5b0-a700-4aa6-a534-c466438dd2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:06.241213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fa5b0-a700-4aa6-a534-c466438dd2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:06.250047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fa5b0-a700-4aa6-a534-c466438dd2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:06.251790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fa5b0-a700-4aa6-a534-c466438dd2c2 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:06.261051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 749/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:08.540627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c05ed1-1ce9-472f-89a6-29e15f0595cb None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.541114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c05ed1-1ce9-472f-89a6-29e15f0595cb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.541208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c05ed1-1ce9-472f-89a6-29e15f0595cb None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.543415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 739/1488] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:08.548969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9b10-c869-4cfb-ab1b-9928b8a3d43d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.549411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9b10-c869-4cfb-ab1b-9928b8a3d43d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.549525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9b10-c869-4cfb-ab1b-9928b8a3d43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.567529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7bcf9b10-c869-4cfb-ab1b-9928b8a3d43d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:08.568717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7bcf9b10-c869-4cfb-ab1b-9928b8a3d43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:08.576515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 750/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:08.616438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c05ed1-1ce9-472f-89a6-29e15f0595cb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:08.628036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c05ed1-1ce9-472f-89a6-29e15f0595cb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:08.631505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4a463-e42a-4477-af90-50396a96fdee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:08.632133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4a463-e42a-4477-af90-50396a96fdee service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:08.632412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4a463-e42a-4477-af90-50396a96fdee service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:08.632682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4a463-e42a-4477-af90-50396a96fdee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:08.641608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4a463-e42a-4477-af90-50396a96fdee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:08.643149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4a463-e42a-4477-af90-50396a96fdee service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:08.652839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 740/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:11.212370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966646-875d-4988-84f6-ad3e5da04a4d None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:11.212935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966646-875d-4988-84f6-ad3e5da04a4d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:11.212935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966646-875d-4988-84f6-ad3e5da04a4d None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:11.214342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 751/1491] 10.222.0.23 () {54 vars in 926 bytes} [Wed Jun 15 14:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:50:11.220260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caa6a7-603b-463b-8bb1-eff22e28f28a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:11.220708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caa6a7-603b-463b-8bb1-eff22e28f28a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:11.220815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caa6a7-603b-463b-8bb1-eff22e28f28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:11.238681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e9caa6a7-603b-463b-8bb1-eff22e28f28a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:50:11.239871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e9caa6a7-603b-463b-8bb1-eff22e28f28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:50:11.247936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 741/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Wed Jun 15 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:50:11.287534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966646-875d-4988-84f6-ad3e5da04a4d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:50:11.299171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966646-875d-4988-84f6-ad3e5da04a4d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:50:11.302756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ac429-dcdf-41f2-87ec-705c96e59e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:50:11.303398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ac429-dcdf-41f2-87ec-705c96e59e15 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:50:11.303678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ac429-dcdf-41f2-87ec-705c96e59e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:50:11.303952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ac429-dcdf-41f2-87ec-705c96e59e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:50:11.313384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ac429-dcdf-41f2-87ec-705c96e59e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:50:11.314873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ac429-dcdf-41f2-87ec-705c96e59e15 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:50:11.325088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 752/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:50:21.233759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=33 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:50:21.320085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=33 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:46.815032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda366da-9d83-4ecd-9724-0adb0cb6c107 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.815589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda366da-9d83-4ecd-9724-0adb0cb6c107 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.815589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda366da-9d83-4ecd-9724-0adb0cb6c107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.833055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-fda366da-9d83-4ecd-9724-0adb0cb6c107 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:46.834268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fda366da-9d83-4ecd-9724-0adb0cb6c107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:46.842092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 742/1494] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:46.851664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813caeb-3240-4338-8019-0614fe27e496 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.852177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813caeb-3240-4338-8019-0614fe27e496 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.852282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813caeb-3240-4338-8019-0614fe27e496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.871746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-0813caeb-3240-4338-8019-0614fe27e496 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:46.873627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0813caeb-3240-4338-8019-0614fe27e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:46.881888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 753/1495] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:46.887155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda366da-9d83-4ecd-9724-0adb0cb6c107 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.897458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda366da-9d83-4ecd-9724-0adb0cb6c107 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.901881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.902570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.902851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.903123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.904442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:51:46.904750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.905813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:46.905813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:51:46.905813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:51:46.906725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:46.918030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 743/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:51:46.927187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813caeb-3240-4338-8019-0614fe27e496 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:46.936402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813caeb-3240-4338-8019-0614fe27e496 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:46.941106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:46.941765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:46.942050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:46.942372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:46.943075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:46.944838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING py.warnings [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:46.944838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: warnings.warn(msg) Jun 15 14:51:46.944838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: Jun 15 14:51:46.946051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.086811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 754/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:51:46 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:51:47.090885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:47.101585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1482f1-5f82-44a4-b0c0-213aa5a3bb55 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:47.160238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:47.160928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.161209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.161487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.162199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:47.163289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:47.163289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:51:47.163289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:51:47.164498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.269305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 744/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:51:47 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:51:47.273575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:47.284733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd8936-e352-41e4-883f-c88183c83206 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:47.289345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:47.290006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.290318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.290594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.291319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:47.292363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING py.warnings [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:47.292363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: warnings.warn(msg) Jun 15 14:51:47.292363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: Jun 15 14:51:47.293256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.306538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 755/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:47.311157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:47.321236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b79b4-3d42-4e56-a784-7f132f79af9f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:47.325713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ea5d4-5022-4f5f-9af6-1e628c80281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:47.326441 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ea5d4-5022-4f5f-9af6-1e628c80281f admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.326730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ea5d4-5022-4f5f-9af6-1e628c80281f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.327011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ea5d4-5022-4f5f-9af6-1e628c80281f admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.327740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ea5d4-5022-4f5f-9af6-1e628c80281f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:47.328754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-4a1ea5d4-5022-4f5f-9af6-1e628c80281f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:47.328754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:51:47.328754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:51:47.329649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ea5d4-5022-4f5f-9af6-1e628c80281f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.343187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 745/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:51:47.347472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:47.357767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0efb94-7df9-4c2d-85c8-a3a998a511da admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:47.362345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f09c8e-93b5-471c-8825-3d9d83d18cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:47.363229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f09c8e-93b5-471c-8825-3d9d83d18cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.363513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f09c8e-93b5-471c-8825-3d9d83d18cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.363791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f09c8e-93b5-471c-8825-3d9d83d18cc3 admin admin] PATH_INFO: `/v3/projects/5eac09dc5cac4695a73f63c387f8bdd7/users/16f12e1176d647c68c360c666244749f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.378924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f09c8e-93b5-471c-8825-3d9d83d18cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eac09dc5cac4695a73f63c387f8bdd7, user_id=16f12e1176d647c68c360c666244749f, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:47.379963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING py.warnings [None req-78f09c8e-93b5-471c-8825-3d9d83d18cc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:47.379963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: warnings.warn(msg) Jun 15 14:51:47.379963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: Jun 15 14:51:47.381904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f09c8e-93b5-471c-8825-3d9d83d18cc3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.476720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 756/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:51:47 2022] PUT /identity/v3/projects/5eac09dc5cac4695a73f63c387f8bdd7/users/16f12e1176d647c68c360c666244749f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:47.489699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadde69-a4bd-43aa-90c4-71788abdf58b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.490004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadde69-a4bd-43aa-90c4-71788abdf58b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.490321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadde69-a4bd-43aa-90c4-71788abdf58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.522086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-bfadde69-a4bd-43aa-90c4-71788abdf58b None None] MFA Rules not processed for user `16f12e1176d647c68c360c666244749f`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:47.523598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bfadde69-a4bd-43aa-90c4-71788abdf58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:47.596886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 746/1502] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:47.610513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1c86f-b2e7-4781-a976-53a821cbb1b3 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.610820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1c86f-b2e7-4781-a976-53a821cbb1b3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.611101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1c86f-b2e7-4781-a976-53a821cbb1b3 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.612506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 757/1503] 10.222.0.23 () {54 vars in 963 bytes} [Wed Jun 15 14:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:51:47.617370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bee58-bfca-4d92-8b4a-237bda7aec74 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.617667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bee58-bfca-4d92-8b4a-237bda7aec74 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.617948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bee58-bfca-4d92-8b4a-237bda7aec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.635970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-206bee58-bfca-4d92-8b4a-237bda7aec74 None None] MFA Rules not processed for user `8d29198c768d4024af04273a3cca8f20`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:47.637158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-206bee58-bfca-4d92-8b4a-237bda7aec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:47.680504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:47.688638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 747/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Wed Jun 15 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:47.696493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1c86f-b2e7-4781-a976-53a821cbb1b3 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:47.707964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1c86f-b2e7-4781-a976-53a821cbb1b3 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:47.711440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:47.712073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.712359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.712629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.720926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:47.722452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:47.731252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 758/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:47.775874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3495d8-7947-4de7-a902-3a439bc9ed63 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.776161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3495d8-7947-4de7-a902-3a439bc9ed63 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.776440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3495d8-7947-4de7-a902-3a439bc9ed63 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.777825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 748/1506] 10.222.0.23 () {52 vars in 829 bytes} [Wed Jun 15 14:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:51:47.781683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:47.842313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35472bc2-e766-4163-a7b9-4dec641dbf47 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:47.846698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:47.847615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:47.847838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:47.848108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] PATH_INFO: `/v3/users/16f12e1176d647c68c360c666244749f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:47.849261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f12e1176d647c68c360c666244749f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:47.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING py.warnings [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:47.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: warnings.warn(msg) Jun 15 14:51:47.850306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: Jun 15 14:51:47.851503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:48.048557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f12e1176d647c68c360c666244749f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:48.049301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f12e1176d647c68c360c666244749f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:48.049908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f12e1176d647c68c360c666244749f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:48.068693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f12e1176d647c68c360c666244749f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:48.150984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f12e1176d647c68c360c666244749f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:48.160670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:48.169825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 759/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:51:47 2022] DELETE /identity/v3/users/16f12e1176d647c68c360c666244749f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:48.174557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:48.175037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:48.175148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:48.192273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:51:48.193461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:48.252785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 749/1508] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:51:48.264034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:48.265535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:48.321911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc5ae0-b546-48cb-adcf-4753000fa1a2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:48.325465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb1045-f5a6-4bd7-b2e8-2bb3b11010b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:48.327112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1045-f5a6-4bd7-b2e8-2bb3b11010b0 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:48.327412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1045-f5a6-4bd7-b2e8-2bb3b11010b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:48.327688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1045-f5a6-4bd7-b2e8-2bb3b11010b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:48.336326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb1045-f5a6-4bd7-b2e8-2bb3b11010b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:48.337769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb1045-f5a6-4bd7-b2e8-2bb3b11010b0 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:48.348845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 760/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:51:48.858269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:51:48.859688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:51:48.872321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eac06f-fd89-4213-be9c-c2e11f3fa3c1 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:51:48.878217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:51:48.879119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:51:48.879369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:51:48.879650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] PATH_INFO: `/v3/projects/5eac09dc5cac4695a73f63c387f8bdd7` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:51:48.880838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eac09dc5cac4695a73f63c387f8bdd7)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:51:48.883389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:51:48.883389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:51:48.883389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:51:48.884624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:51:49.006987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eac09dc5cac4695a73f63c387f8bdd7', 'request_id': 'req-c528776e-52c8-442a-abe6-ba23edb4520f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:49.021579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eac09dc5cac4695a73f63c387f8bdd7', 'request_id': 'req-c528776e-52c8-442a-abe6-ba23edb4520f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:49.105168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eac09dc5cac4695a73f63c387f8bdd7', 'request_id': 'req-c528776e-52c8-442a-abe6-ba23edb4520f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:49.116341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] The token cache is being invalidate because project 5eac09dc5cac4695a73f63c387f8bdd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:51:49.116626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:49.117335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c528776e-52c8-442a-abe6-ba23edb4520f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:51:49.118924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 750/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:51:48 2022] DELETE /identity/v3/projects/5eac09dc5cac4695a73f63c387f8bdd7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:51:58.344750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=29 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:51:59.113143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:31.730872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369dc4a-9174-43f8-976d-d8938af70d97 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:31.731462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369dc4a-9174-43f8-976d-d8938af70d97 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:31.731462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369dc4a-9174-43f8-976d-d8938af70d97 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:31.732833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:31.738442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1e3cf-8c18-4af0-8c78-a5e5d0350fc1 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:31.738899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1e3cf-8c18-4af0-8c78-a5e5d0350fc1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:31.739008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1e3cf-8c18-4af0-8c78-a5e5d0350fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:31.757580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-d3d1e3cf-8c18-4af0-8c78-a5e5d0350fc1 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:31.758830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d3d1e3cf-8c18-4af0-8c78-a5e5d0350fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:31.818373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:31.988821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369dc4a-9174-43f8-976d-d8938af70d97 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:31.993805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1369dc4a-9174-43f8-976d-d8938af70d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:32.064025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369dc4a-9174-43f8-976d-d8938af70d97 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:32.069415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a668ec-a443-4b9d-84ed-4e0841f898a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:32.069998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a668ec-a443-4b9d-84ed-4e0841f898a6 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:32.070315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a668ec-a443-4b9d-84ed-4e0841f898a6 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:32.070595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a668ec-a443-4b9d-84ed-4e0841f898a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:32.078707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a668ec-a443-4b9d-84ed-4e0841f898a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:32.080321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a668ec-a443-4b9d-84ed-4e0841f898a6 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:32.089528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 762/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:32.977031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396989c-0958-4fb4-9649-0f34d7d38375 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:32.977599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396989c-0958-4fb4-9649-0f34d7d38375 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:32.977599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396989c-0958-4fb4-9649-0f34d7d38375 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:32.979038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:32.984316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160bf51-5eac-4119-a001-c7cac35af59b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:32.984316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160bf51-5eac-4119-a001-c7cac35af59b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:32.984900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160bf51-5eac-4119-a001-c7cac35af59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:33.003325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a160bf51-5eac-4119-a001-c7cac35af59b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:33.004657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a160bf51-5eac-4119-a001-c7cac35af59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:33.013298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:33.166060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396989c-0958-4fb4-9649-0f34d7d38375 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:33.181376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396989c-0958-4fb4-9649-0f34d7d38375 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:33.186798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eea314-e8a4-42e7-a277-afeff8bb8c08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:33.187448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eea314-e8a4-42e7-a277-afeff8bb8c08 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:33.187728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eea314-e8a4-42e7-a277-afeff8bb8c08 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:33.188004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eea314-e8a4-42e7-a277-afeff8bb8c08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:33.196152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eea314-e8a4-42e7-a277-afeff8bb8c08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:33.197791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eea314-e8a4-42e7-a277-afeff8bb8c08 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:33.208015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 753/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:34.394867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f35e3c-cec6-4707-b11a-31e383307f5a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:34.395431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f35e3c-cec6-4707-b11a-31e383307f5a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:34.395527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f35e3c-cec6-4707-b11a-31e383307f5a None None] PATH_INFO: `/v3` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:34.396884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 764/1517] 10.222.0.55 () {54 vars in 925 bytes} [Wed Jun 15 14:53:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:53:34.406724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee779f30-fc26-4cb9-a928-2d65770a2141 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:34.407132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee779f30-fc26-4cb9-a928-2d65770a2141 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:34.407279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee779f30-fc26-4cb9-a928-2d65770a2141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:34.425553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ee779f30-fc26-4cb9-a928-2d65770a2141 None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:34.426786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ee779f30-fc26-4cb9-a928-2d65770a2141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:34.489106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 754/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Jun 15 14:53:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:42.997090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=37 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:44.466075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95e87-8380-44f7-91f1-f677eb64afaa None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:44.466916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95e87-8380-44f7-91f1-f677eb64afaa None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:44.466916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95e87-8380-44f7-91f1-f677eb64afaa None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:44.468042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 765/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:44.472902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cc6b4-e759-4290-b413-07646f0acb16 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:44.473301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cc6b4-e759-4290-b413-07646f0acb16 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:44.473450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cc6b4-e759-4290-b413-07646f0acb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:44.618340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ae5cc6b4-e759-4290-b413-07646f0acb16 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:44.619555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ae5cc6b4-e759-4290-b413-07646f0acb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:44.628462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 755/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:44.784608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95e87-8380-44f7-91f1-f677eb64afaa None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:44.799839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95e87-8380-44f7-91f1-f677eb64afaa None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:44.805268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d4d87-9c0e-4b3c-b130-74913e4beecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:44.808433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d4d87-9c0e-4b3c-b130-74913e4beecc service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:44.808433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d4d87-9c0e-4b3c-b130-74913e4beecc service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:44.808433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d4d87-9c0e-4b3c-b130-74913e4beecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:44.815214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d4d87-9c0e-4b3c-b130-74913e4beecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:44.816894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d4d87-9c0e-4b3c-b130-74913e4beecc service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:44.826595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 766/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:45.713188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aeb34-aa08-4d1b-ab48-a2206d5e0596 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:45.713769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aeb34-aa08-4d1b-ab48-a2206d5e0596 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:45.713769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aeb34-aa08-4d1b-ab48-a2206d5e0596 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:45.715112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 14:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:45.720120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ce9e-e2f8-4c46-88f4-4b3f7b69bf2e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:45.720526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ce9e-e2f8-4c46-88f4-4b3f7b69bf2e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:45.720725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ce9e-e2f8-4c46-88f4-4b3f7b69bf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:45.744450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-8172ce9e-e2f8-4c46-88f4-4b3f7b69bf2e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:45.745994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8172ce9e-e2f8-4c46-88f4-4b3f7b69bf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:45.755378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:45.911476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31aeb34-aa08-4d1b-ab48-a2206d5e0596 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:45.927290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31aeb34-aa08-4d1b-ab48-a2206d5e0596 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:45.932615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef96dd2-a835-4d45-8ea6-17ecd2f8dd55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:45.936218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef96dd2-a835-4d45-8ea6-17ecd2f8dd55 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:45.936218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef96dd2-a835-4d45-8ea6-17ecd2f8dd55 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:45.936218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef96dd2-a835-4d45-8ea6-17ecd2f8dd55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:45.942818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef96dd2-a835-4d45-8ea6-17ecd2f8dd55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:45.944424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef96dd2-a835-4d45-8ea6-17ecd2f8dd55 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:45.954659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 757/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:46.016526 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772da02-375b-4b9a-b1b6-c499b5d6523a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:46.027941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772da02-375b-4b9a-b1b6-c499b5d6523a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:46.027941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772da02-375b-4b9a-b1b6-c499b5d6523a None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:46.027941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 768/1525] 10.222.0.23 () {54 vars in 960 bytes} [Wed Jun 15 14:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:46.028263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1d50e-5c3a-49ab-a2c2-ba87da1b224d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:46.028263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1d50e-5c3a-49ab-a2c2-ba87da1b224d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:46.028263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1d50e-5c3a-49ab-a2c2-ba87da1b224d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:46.049119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a1b1d50e-5c3a-49ab-a2c2-ba87da1b224d None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:46.050833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a1b1d50e-5c3a-49ab-a2c2-ba87da1b224d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:46.062061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 758/1526] 10.222.0.23 () {58 vars in 1057 bytes} [Wed Jun 15 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:46.069581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772da02-375b-4b9a-b1b6-c499b5d6523a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:46.083797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772da02-375b-4b9a-b1b6-c499b5d6523a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:46.089287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94ea6e-4e5f-4f1b-8db6-8de3c06bcf31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:46.089957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ea6e-4e5f-4f1b-8db6-8de3c06bcf31 service placement] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:46.091636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ea6e-4e5f-4f1b-8db6-8de3c06bcf31 service placement] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:46.091937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ea6e-4e5f-4f1b-8db6-8de3c06bcf31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:46.102017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94ea6e-4e5f-4f1b-8db6-8de3c06bcf31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:46.103663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94ea6e-4e5f-4f1b-8db6-8de3c06bcf31 service placement] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:46.113486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 769/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:53:49.936077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aac69-a29c-435e-83c1-dd8cc33bd393 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:49.936616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aac69-a29c-435e-83c1-dd8cc33bd393 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:49.936616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aac69-a29c-435e-83c1-dd8cc33bd393 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:49.939050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:53:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:53:49.944273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242ad3c-73c8-468a-ace8-7f4d259dba3a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:49.944706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242ad3c-73c8-468a-ace8-7f4d259dba3a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:49.944821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242ad3c-73c8-468a-ace8-7f4d259dba3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:49.965145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e242ad3c-73c8-468a-ace8-7f4d259dba3a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:49.966374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e242ad3c-73c8-468a-ace8-7f4d259dba3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:49.974463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:50.011056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aac69-a29c-435e-83c1-dd8cc33bd393 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:50.012744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-076aac69-a29c-435e-83c1-dd8cc33bd393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:50.070817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aac69-a29c-435e-83c1-dd8cc33bd393 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:50.074431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8083b19-ff62-4164-aaff-a16ae1c1b07e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:50.075075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083b19-ff62-4164-aaff-a16ae1c1b07e service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:50.075350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083b19-ff62-4164-aaff-a16ae1c1b07e service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:50.075618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8083b19-ff62-4164-aaff-a16ae1c1b07e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:50.085514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8083b19-ff62-4164-aaff-a16ae1c1b07e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:50.087008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8083b19-ff62-4164-aaff-a16ae1c1b07e service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:50.092649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:53:50.098105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 760/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:52.255154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19c5e-04e9-4b9f-b6b6-9e467add1e7c None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.255741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19c5e-04e9-4b9f-b6b6-9e467add1e7c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.255741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19c5e-04e9-4b9f-b6b6-9e467add1e7c None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.257138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:52.262475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf899c2-4e65-4362-bf35-63dcfa6b0919 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.262904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf899c2-4e65-4362-bf35-63dcfa6b0919 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.263009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf899c2-4e65-4362-bf35-63dcfa6b0919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.281604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-dbf899c2-4e65-4362-bf35-63dcfa6b0919 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:52.282843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dbf899c2-4e65-4362-bf35-63dcfa6b0919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:52.291264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:52.328492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f19c5e-04e9-4b9f-b6b6-9e467add1e7c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:52.330110 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-74f19c5e-04e9-4b9f-b6b6-9e467add1e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:52.343834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f19c5e-04e9-4b9f-b6b6-9e467add1e7c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:52.347898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1111bf1-1eb0-444d-9936-07e746a0d5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:52.348536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111bf1-1eb0-444d-9936-07e746a0d5dc service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:52.348817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111bf1-1eb0-444d-9936-07e746a0d5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:52.349090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111bf1-1eb0-444d-9936-07e746a0d5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:52.358120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1111bf1-1eb0-444d-9936-07e746a0d5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:52.364386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1111bf1-1eb0-444d-9936-07e746a0d5dc service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:52.369435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 772/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:56.110096 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eeb37-0e86-4599-b1fd-c8f1fb31f87a None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.110721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eeb37-0e86-4599-b1fd-c8f1fb31f87a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.110721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eeb37-0e86-4599-b1fd-c8f1fb31f87a None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.111981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:56.117020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344c344-88b0-49ec-8e5d-87877ce33f97 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.117431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344c344-88b0-49ec-8e5d-87877ce33f97 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.117669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344c344-88b0-49ec-8e5d-87877ce33f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.136201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6344c344-88b0-49ec-8e5d-87877ce33f97 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:56.137381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6344c344-88b0-49ec-8e5d-87877ce33f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:56.146497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:56.418661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eeb37-0e86-4599-b1fd-c8f1fb31f87a None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:56.420351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0d4eeb37-0e86-4599-b1fd-c8f1fb31f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:56.478337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eeb37-0e86-4599-b1fd-c8f1fb31f87a None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:56.481937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706337b-dd68-4947-bb73-727d3debe2e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:56.482692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706337b-dd68-4947-bb73-727d3debe2e2 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:56.482976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706337b-dd68-4947-bb73-727d3debe2e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:56.483253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706337b-dd68-4947-bb73-727d3debe2e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:56.496690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706337b-dd68-4947-bb73-727d3debe2e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:56.498980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706337b-dd68-4947-bb73-727d3debe2e2 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:56.514272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 763/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:53:58.705932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7bfd-b4bf-4866-bde1-a164d18d982f None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:58.706534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7bfd-b4bf-4866-bde1-a164d18d982f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:58.706534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7bfd-b4bf-4866-bde1-a164d18d982f None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:58.707896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 14:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:53:58.713014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937115d9-3cac-4c7e-bb9e-9efa17415d3f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:58.713419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937115d9-3cac-4c7e-bb9e-9efa17415d3f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:58.713565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937115d9-3cac-4c7e-bb9e-9efa17415d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:58.731858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-937115d9-3cac-4c7e-bb9e-9efa17415d3f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:53:58.733066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-937115d9-3cac-4c7e-bb9e-9efa17415d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:58.741024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:53:59.005433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de7bfd-b4bf-4866-bde1-a164d18d982f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:53:59.007314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b1de7bfd-b4bf-4866-bde1-a164d18d982f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:53:59.021015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de7bfd-b4bf-4866-bde1-a164d18d982f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:53:59.024463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3feabe4-45e9-4773-a779-ff95a6c67125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:53:59.025093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3feabe4-45e9-4773-a779-ff95a6c67125 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:53:59.025371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3feabe4-45e9-4773-a779-ff95a6c67125 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:53:59.025637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3feabe4-45e9-4773-a779-ff95a6c67125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:53:59.034372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3feabe4-45e9-4773-a779-ff95a6c67125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:53:59.035803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3feabe4-45e9-4773-a779-ff95a6c67125 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:53:59.046255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 775/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:54:08.727253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=32 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:54:09.042122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=33 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:30.230195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96b28a-136f-41cb-9046-46a7b7b89642 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.231612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96b28a-136f-41cb-9046-46a7b7b89642 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.232126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96b28a-136f-41cb-9046-46a7b7b89642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.260713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1a96b28a-136f-41cb-9046-46a7b7b89642 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:30.262117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1a96b28a-136f-41cb-9046-46a7b7b89642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.274389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 765/1540] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:30.284822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a036dde-4fdf-4d71-918b-9563328de35f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.285419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a036dde-4fdf-4d71-918b-9563328de35f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.285419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a036dde-4fdf-4d71-918b-9563328de35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.311241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3a036dde-4fdf-4d71-918b-9563328de35f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:30.313357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3a036dde-4fdf-4d71-918b-9563328de35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.324085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 776/1541] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:30.328839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96b28a-136f-41cb-9046-46a7b7b89642 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:30.339796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5ada3-e13f-4d48-8d75-960eeb93d600 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.340157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5ada3-e13f-4d48-8d75-960eeb93d600 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.340371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5ada3-e13f-4d48-8d75-960eeb93d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.341639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96b28a-136f-41cb-9046-46a7b7b89642 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:30.346813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:30.347525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.347812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.348087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.348845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:30.349154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:30.350909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:30.371514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-58d5ada3-e13f-4d48-8d75-960eeb93d600 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:30.372772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-58d5ada3-e13f-4d48-8d75-960eeb93d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.373838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 766/1542] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:30.382829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:30.388052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 777/1543] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:30.403833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c0865-978f-4770-a0c9-da6e2b32a131 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.404183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c0865-978f-4770-a0c9-da6e2b32a131 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.404430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c0865-978f-4770-a0c9-da6e2b32a131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.406348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597a0b1-21b4-44d9-a9ac-9781264d16b5 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:30.412188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:30.414389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.414389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.414389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.414389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:30.423493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:55:30.423493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:55:30.423493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:55:30.424651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:30.436676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-612c0865-978f-4770-a0c9-da6e2b32a131 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:30.437969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-612c0865-978f-4770-a0c9-da6e2b32a131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.455200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 778/1544] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:55:30.472078 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c0865-978f-4770-a0c9-da6e2b32a131 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:30.477338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c0865-978f-4770-a0c9-da6e2b32a131 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:30.487339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:30.487901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.488195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.488479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.489349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:30.489667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:30.490793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING py.warnings [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:55:30.490793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: warnings.warn(msg) Jun 15 14:55:30.490793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: Jun 15 14:55:30.491718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:30.516452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 779/1545] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:30.531778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:30.555605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049c3ad-36e3-44cc-96f2-ee2b72c52067 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:30.565391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82b2e7-f5a8-4236-9fae-cc35e765162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:30.566083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82b2e7-f5a8-4236-9fae-cc35e765162f admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.574300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82b2e7-f5a8-4236-9fae-cc35e765162f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.574300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82b2e7-f5a8-4236-9fae-cc35e765162f admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.574300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82b2e7-f5a8-4236-9fae-cc35e765162f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:30.574300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82b2e7-f5a8-4236-9fae-cc35e765162f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:30.615052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 767/1546] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/projects => generated 375 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:30.619549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:30.727662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4193b-6d3b-41dd-b9e7-4febcdd8d9f5 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:30.734145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fb1b9-0795-435e-95f9-5fdb31c035c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:30.735306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fb1b9-0795-435e-95f9-5fdb31c035c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.735901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fb1b9-0795-435e-95f9-5fdb31c035c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.736413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fb1b9-0795-435e-95f9-5fdb31c035c0 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.737466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fb1b9-0795-435e-95f9-5fdb31c035c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:30.739804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fb1b9-0795-435e-95f9-5fdb31c035c0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:30.781186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 780/1547] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/projects => generated 379 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:30.785873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b8cdd-7642-4279-b1dc-e534d960d0b4 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.786196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b8cdd-7642-4279-b1dc-e534d960d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.786501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b8cdd-7642-4279-b1dc-e534d960d0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.807565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-0c7b8cdd-7642-4279-b1dc-e534d960d0b4 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:30.808835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0c7b8cdd-7642-4279-b1dc-e534d960d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.877795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 781/1548] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:30.879262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662d48-b06a-4d8e-843e-d0ac931235c7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.879557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662d48-b06a-4d8e-843e-d0ac931235c7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.879836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662d48-b06a-4d8e-843e-d0ac931235c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.898007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1f662d48-b06a-4d8e-843e-d0ac931235c7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:30.899252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1f662d48-b06a-4d8e-843e-d0ac931235c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.907107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 782/1549] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:30.908518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c050d5-d807-4922-9a37-635734312f17 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.908810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c050d5-d807-4922-9a37-635734312f17 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.909085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c050d5-d807-4922-9a37-635734312f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.926684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-02c050d5-d807-4922-9a37-635734312f17 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:30.942502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-02c050d5-d807-4922-9a37-635734312f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:30.942502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 783/1550] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:30.942502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c050d5-d807-4922-9a37-635734312f17 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:30.947236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c050d5-d807-4922-9a37-635734312f17 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:30.951722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd79bf-4262-4ad9-922f-c1a1f470a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:30.952370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd79bf-4262-4ad9-922f-c1a1f470a01f admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.952656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd79bf-4262-4ad9-922f-c1a1f470a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.952932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd79bf-4262-4ad9-922f-c1a1f470a01f admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:30.953615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd79bf-4262-4ad9-922f-c1a1f470a01f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:30.954780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING py.warnings [None req-7ccd79bf-4262-4ad9-922f-c1a1f470a01f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:55:30.954780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: warnings.warn(msg) Jun 15 14:55:30.954780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: Jun 15 14:55:30.956183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd79bf-4262-4ad9-922f-c1a1f470a01f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:30.984805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 768/1551] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/users => generated 288 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:55:30.986626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93aaa07-9cd3-4d5e-9a8f-52b940bef218 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:30.987707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93aaa07-9cd3-4d5e-9a8f-52b940bef218 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:30.987707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93aaa07-9cd3-4d5e-9a8f-52b940bef218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.005200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e93aaa07-9cd3-4d5e-9a8f-52b940bef218 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.006474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e93aaa07-9cd3-4d5e-9a8f-52b940bef218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.014210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 769/1552] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.015410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5378b-df0e-4a7a-8c66-a5b3e348406d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.015699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5378b-df0e-4a7a-8c66-a5b3e348406d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.015982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5378b-df0e-4a7a-8c66-a5b3e348406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.033847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5bd5378b-df0e-4a7a-8c66-a5b3e348406d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.035091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5bd5378b-df0e-4a7a-8c66-a5b3e348406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.042934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 770/1553] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.044129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dca90-254e-46c5-92ea-711e3cfc3491 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.044418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dca90-254e-46c5-92ea-711e3cfc3491 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.044695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dca90-254e-46c5-92ea-711e3cfc3491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.063039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-431dca90-254e-46c5-92ea-711e3cfc3491 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.071743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-431dca90-254e-46c5-92ea-711e3cfc3491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.072192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 771/1554] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.073544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7c18-6d4f-456b-8ea9-22fd93c9b3d9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.073836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7c18-6d4f-456b-8ea9-22fd93c9b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.074157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7c18-6d4f-456b-8ea9-22fd93c9b3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.093088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2c1e7c18-6d4f-456b-8ea9-22fd93c9b3d9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.094322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2c1e7c18-6d4f-456b-8ea9-22fd93c9b3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.103286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 772/1555] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.103961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 784/1556] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:30 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:31.104561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521bb0e-835c-480c-a300-8d85ca476cda None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.104978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521bb0e-835c-480c-a300-8d85ca476cda None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.105265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521bb0e-835c-480c-a300-8d85ca476cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.105907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd7710-c68e-4725-a6c5-36dba7efeaed None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.106382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd7710-c68e-4725-a6c5-36dba7efeaed None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.106820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd7710-c68e-4725-a6c5-36dba7efeaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.124017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c521bb0e-835c-480c-a300-8d85ca476cda None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.125226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-81cd7710-c68e-4725-a6c5-36dba7efeaed None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.125363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c521bb0e-835c-480c-a300-8d85ca476cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.126583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-81cd7710-c68e-4725-a6c5-36dba7efeaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.133213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 773/1557] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.133969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521bb0e-835c-480c-a300-8d85ca476cda None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.138425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 785/1558] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.139681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2646028-01a0-43c5-81fc-6f3aa1d18b4d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.139966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2646028-01a0-43c5-81fc-6f3aa1d18b4d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.140241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2646028-01a0-43c5-81fc-6f3aa1d18b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.147887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521bb0e-835c-480c-a300-8d85ca476cda None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.152641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fc189-e48b-4cb7-9767-bb82e8b2d7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.153309 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc189-e48b-4cb7-9767-bb82e8b2d7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.153597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc189-e48b-4cb7-9767-bb82e8b2d7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.153905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fc189-e48b-4cb7-9767-bb82e8b2d7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.154671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fc189-e48b-4cb7-9767-bb82e8b2d7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.156302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fc189-e48b-4cb7-9767-bb82e8b2d7f5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.160139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a2646028-01a0-43c5-81fc-6f3aa1d18b4d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.169148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a2646028-01a0-43c5-81fc-6f3aa1d18b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.169282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 774/1559] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:31.169282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcbc1d-4ba8-4a7a-9c9b-d11c4727929a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.169282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcbc1d-4ba8-4a7a-9c9b-d11c4727929a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.169282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcbc1d-4ba8-4a7a-9c9b-d11c4727929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.170099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 786/1560] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.173688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8b00-f1f3-49b6-9bde-0be6557c6c90 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.173688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8b00-f1f3-49b6-9bde-0be6557c6c90 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.173688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8b00-f1f3-49b6-9bde-0be6557c6c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.190528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b05b8b00-f1f3-49b6-9bde-0be6557c6c90 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.191528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-f9bcbc1d-4ba8-4a7a-9c9b-d11c4727929a None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:31.191723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b05b8b00-f1f3-49b6-9bde-0be6557c6c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.193003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f9bcbc1d-4ba8-4a7a-9c9b-d11c4727929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:31.199556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 787/1561] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.200402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8b00-f1f3-49b6-9bde-0be6557c6c90 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.217076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 775/1562] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:31.217076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcbc1d-4ba8-4a7a-9c9b-d11c4727929a None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.217282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8b00-f1f3-49b6-9bde-0be6557c6c90 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.217372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcbc1d-4ba8-4a7a-9c9b-d11c4727929a None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.217372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.217554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.218412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.218535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.218743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.219027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.219774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:31.219959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.220124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.220281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.221174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.221688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.222889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.233575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 776/1563] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:31.234349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 788/1564] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:31.234530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.235603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.244290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b797a4f-6d94-452d-b0c9-a8daeb67f857 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.246586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a4333-b7a9-418e-823e-e819bc8a927b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.248820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.252331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.252331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.252331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.252331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:31.252331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.252816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.252941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.253210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.253488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.253585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.254243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:31.254559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.256183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.265506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 777/1565] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:31.266337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.283615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 789/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:31.283838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49809fbc-c8d3-4766-9dbf-e458aeeccde0 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.283948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.284056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.284056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.284056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.284056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.284412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.285023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.289070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cd70a-edcb-4a61-8906-8d7b897f7485 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.293971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.295212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.295212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.296228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.296228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:31.296400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.300389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 778/1567] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:31.300389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.300638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.310340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77a93d-0a8d-4b3d-9dcc-a361cf90d624 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.314757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 790/1568] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:31.314757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.316080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.316802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.317225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.317578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.318415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:31.318789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.320524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.323537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa167715-fefa-47ad-8edc-e9773e621cfc admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.328264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.328924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.329302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.330680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.330895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:31.331209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.332811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.333668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 779/1569] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:31.334294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.347130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 791/1570] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:31.347130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.360448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210468a2-5770-4d7f-a529-04107eb5d3b7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.360554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:31.360672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.360769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538f6a-36cc-49ca-8740-7efaf7e7977e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.360863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.360863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.360863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.360863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.360863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.362260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:31.365247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.365912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.366251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.366514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.367321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.369242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.370459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 780/1571] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:31.370459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.382886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662b416-a25f-446c-8458-0944a745fe1e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.387371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.388025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.388304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.388607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.389290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.391342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.551979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 792/1572] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/projects => generated 405 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:31.553057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.581159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 781/1573] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/projects => generated 371 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:31.581974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.617369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7ca49-ea6a-4a92-921d-6f1738ddb16f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.646708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669242bd-e82c-4f29-86f2-da38eeeaed81 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.652010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.652902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.653179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.653455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] PATH_INFO: `/v3/projects/3ae6c02cace9481d9b87e8f9d61323c3/users/df400aee39a3498d8a26d55180075ab6/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.668794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae6c02cace9481d9b87e8f9d61323c3, user_id=df400aee39a3498d8a26d55180075ab6, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.669844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:55:31.669844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:55:31.669844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:55:31.671836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.676284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.676956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.677238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.677510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.678312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.680245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.756184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 782/1574] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:31 2022] PUT /identity/v3/projects/3ae6c02cace9481d9b87e8f9d61323c3/users/df400aee39a3498d8a26d55180075ab6/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:31.757105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.807958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 793/1575] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/projects => generated 393 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:31.810007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:31.818809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31eae6e-a423-40d2-ae4a-008fda3f87f9 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.878344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12685000-e809-4e31-8a49-b2ecbf613225 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:31.880601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.881272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.881550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.881819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.882542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.883275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:31.883936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:31.884214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:31.884455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.884570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:31.885157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:31.887098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:31.893722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:31.997078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 783/1576] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/projects => generated 401 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:31.998040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.047291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:32.053934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 794/1577] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:32.060284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.060444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da065db-a4c6-4d32-addf-c9ae95091832 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.119799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.120577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.121181 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.121634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.122978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.124985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.127360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ebd5-04b8-4cc5-bfd5-c727a7e28877 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.132327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.140258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.140258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.140258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] PATH_INFO: `/v3/projects/dbb7af9fb2864248800c775d2d0d174e/users/5fdf869a706a445d950ee8f59595ad4a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.150051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb7af9fb2864248800c775d2d0d174e, user_id=5fdf869a706a445d950ee8f59595ad4a, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.152741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.396605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 795/1578] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:32 2022] PUT /identity/v3/projects/dbb7af9fb2864248800c775d2d0d174e/users/5fdf869a706a445d950ee8f59595ad4a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:32.397718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.399344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 784/1579] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:31 2022] POST /identity/v3/projects => generated 387 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:32.400497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.462486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f35d8b-afc9-4b50-a548-36a204f21557 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.463449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce65786-a097-47c2-b0d5-829c6c778e22 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.467525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b755d6d-741e-43c9-beee-aa569f409d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.467994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.468201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b755d6d-741e-43c9-beee-aa569f409d84 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.468484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b755d6d-741e-43c9-beee-aa569f409d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.468657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.468765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b755d6d-741e-43c9-beee-aa569f409d84 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.468940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.469215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.469440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b755d6d-741e-43c9-beee-aa569f409d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.469894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.471696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b755d6d-741e-43c9-beee-aa569f409d84 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.472578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.631772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 785/1580] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:32.635600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 796/1581] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:32.635757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f02c-f6d6-478b-9a76-429ee2ca03e7 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.635865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.635954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f02c-f6d6-478b-9a76-429ee2ca03e7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.635954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f02c-f6d6-478b-9a76-429ee2ca03e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.645820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a92f74-1bef-4c08-8901-d3c832ed3a1c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.650719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.651323 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.651602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.651878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.652570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.653218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-81e4f02c-f6d6-478b-9a76-429ee2ca03e7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:32.654469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-81e4f02c-f6d6-478b-9a76-429ee2ca03e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:32.654589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.663320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 786/1582] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:32.663739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4f02c-f6d6-478b-9a76-429ee2ca03e7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.669433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:32.673252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4f02c-f6d6-478b-9a76-429ee2ca03e7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.678479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6562e-fd39-496a-a073-935bdb28222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.679153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6562e-fd39-496a-a073-935bdb28222c admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.679435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6562e-fd39-496a-a073-935bdb28222c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.679710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6562e-fd39-496a-a073-935bdb28222c admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.680400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6562e-fd39-496a-a073-935bdb28222c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.682342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6562e-fd39-496a-a073-935bdb28222c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.796369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 797/1583] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:32.797442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.806984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9344b3-1aea-431e-850d-d276febbe948 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.811460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.812151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.812428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.812700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.813369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.816191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.832176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 787/1584] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/users => generated 301 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:32.833556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42bb38-4ad4-4885-bf8c-a99bac5162c9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.833876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42bb38-4ad4-4885-bf8c-a99bac5162c9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.834152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42bb38-4ad4-4885-bf8c-a99bac5162c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.850630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1b42bb38-4ad4-4885-bf8c-a99bac5162c9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:32.851813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1b42bb38-4ad4-4885-bf8c-a99bac5162c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:32.859338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 788/1585] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:32.860067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42bb38-4ad4-4885-bf8c-a99bac5162c9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.869001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42bb38-4ad4-4885-bf8c-a99bac5162c9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.873504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.874144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.874453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.874728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.875402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.877957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:32.973907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 798/1586] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:32.974925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:32.984501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3288e7-7e6e-4b28-949d-a55475b24893 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:32.989648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:32.990327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:32.990613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:32.990889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:32.991611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:32.993233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.003867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 799/1587] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.004697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.013736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be519e-40ce-41db-8956-4e266f98184e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.019072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.019693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.019975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.020249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.020954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.022858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.032471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 800/1588] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.033947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.050380 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:33.081344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 789/1589] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:32 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:33.082612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.086591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9d2b0-b3d1-424c-af25-4215705fc4c3 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.105312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.105312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.105312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.105312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.105709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc01834-d926-42c0-9fa0-7029eaba604a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.105709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.105709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.105709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.106060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.106145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.106277 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.106366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.106452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.112131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 801/1590] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:33.113012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.117661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 790/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.117661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.123394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44253625-400a-409b-9e5e-4ae457364b26 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.128337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2186cf3-6d76-42ea-af79-44ece401fe63 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.128630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.129483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.129883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.130285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.131084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.132209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.132849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.132959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.133420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.133420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.143647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 802/1592] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.143808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.144655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.145272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.155005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639147b-515a-45ba-bd11-6790167ad8ce admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.155561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 791/1593] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:33.156355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.165874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.179770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.179944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d481-aaa7-4449-a82f-e7f8908e34fd service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.180055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.180055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.180055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.180055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.180392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.180392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.180392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.180392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.180725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.180725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.181354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 803/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.182355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.192028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 792/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.192537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95590357-2b6f-4c37-afba-8a9b8cada7a5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.192848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.199447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.200109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.200389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.200664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.201395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.203042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301ee55-2e33-481d-9d17-0a2fe95b9d3a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.203156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.208575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.209290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.209571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.209871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.210632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.212282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.214065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 804/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.214960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.225582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1fe9-a0fc-4182-a0d9-c4957514999b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.226622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 793/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.227330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.233274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.233939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.234244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.234520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.235243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.236712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b7dfd-5a67-470a-ae6b-00dab66f35ae admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.237239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.241283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.241927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.242238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.242518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.243238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.244813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.248010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 805/1598] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.251219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.257966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 794/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.257966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.261405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade6b35-dc92-496d-9239-a013fdd09e34 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.265841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a4416-bd75-426d-912f-557d238e0fcf admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.268729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.269279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.269569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.270000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.270771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.271516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.272657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.272777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.272930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.273205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] PATH_INFO: `/v3/projects/6880552302ad4736b37c41a389584c65/users/c83a81bc7e6749239b654f0b1992aab8/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.287176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 806/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.287176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.307516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6880552302ad4736b37c41a389584c65, user_id=c83a81bc7e6749239b654f0b1992aab8, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.307516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.307718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1b56-1c38-47a0-b99f-e92a5f80976e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.307718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.307718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.307718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.308043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] PATH_INFO: `/v3/projects/afc5256083884a55982634bb457f7dba/users/e0d94fbf267b4152920e0fd30c929740/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.323036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc5256083884a55982634bb457f7dba, user_id=e0d94fbf267b4152920e0fd30c929740, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.325752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.579209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 795/1601] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:33 2022] PUT /identity/v3/projects/6880552302ad4736b37c41a389584c65/users/c83a81bc7e6749239b654f0b1992aab8/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:33.580859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.643840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55ba01-9bfd-4672-ada7-b1f3202c667f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.648693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.649359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.649647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.649925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.650685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.652364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.662673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 796/1602] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:33.663432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.672556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae07c9-55a8-4318-b900-21ddeea301af admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.677771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.678683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.678967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.679245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] PATH_INFO: `/v3/projects/65a03d17e6f54c66bb4ca7571246be87/users/cbc9d5ce4a3241ea898f216543114fdf/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.694399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 807/1603] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:33 2022] PUT /identity/v3/projects/afc5256083884a55982634bb457f7dba/users/e0d94fbf267b4152920e0fd30c929740/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:33.695391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.706219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a03d17e6f54c66bb4ca7571246be87, user_id=cbc9d5ce4a3241ea898f216543114fdf, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.706219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.763929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5f31-aa1a-479a-91d7-6f63332028d6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:33.768681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa74e2-8a01-49c5-bffc-1f20e8c0f46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:33.769586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa74e2-8a01-49c5-bffc-1f20e8c0f46e admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:33.769867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa74e2-8a01-49c5-bffc-1f20e8c0f46e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:33.770141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa74e2-8a01-49c5-bffc-1f20e8c0f46e admin admin] PATH_INFO: `/v3/projects/44a5d9a7cb35407f8accbaff750ba980/users/082433ca0c3947efa367a562cb2802c1/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:33.785204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa74e2-8a01-49c5-bffc-1f20e8c0f46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a5d9a7cb35407f8accbaff750ba980, user_id=082433ca0c3947efa367a562cb2802c1, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:33.787916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa74e2-8a01-49c5-bffc-1f20e8c0f46e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:33.935229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 797/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:33 2022] PUT /identity/v3/projects/65a03d17e6f54c66bb4ca7571246be87/users/cbc9d5ce4a3241ea898f216543114fdf/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:33.936093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:33.949213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:33.997014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a33c8d-305f-43d2-b375-a149af062535 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.000808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 808/1605] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:33 2022] PUT /identity/v3/projects/44a5d9a7cb35407f8accbaff750ba980/users/082433ca0c3947efa367a562cb2802c1/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:34.002547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa77325-599f-4dac-b6fe-eb7522602844 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.002843 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa77325-599f-4dac-b6fe-eb7522602844 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.003139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa77325-599f-4dac-b6fe-eb7522602844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.063678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-dfa77325-599f-4dac-b6fe-eb7522602844 None None] MFA Rules not processed for user `c83a81bc7e6749239b654f0b1992aab8`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:34.063678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dfa77325-599f-4dac-b6fe-eb7522602844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:34.063678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:34.064197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc897b9-dbf8-4f91-a292-cd6b1d8bc5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.064197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc897b9-dbf8-4f91-a292-cd6b1d8bc5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.064197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc897b9-dbf8-4f91-a292-cd6b1d8bc5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.064197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc897b9-dbf8-4f91-a292-cd6b1d8bc5d1 admin admin] PATH_INFO: `/v3/projects/c100074063c44d96adf93e0b0f080527/users/e367e391e68a477d861a515f846a8dfa/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.070421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc897b9-dbf8-4f91-a292-cd6b1d8bc5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c100074063c44d96adf93e0b0f080527, user_id=e367e391e68a477d861a515f846a8dfa, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.073093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc897b9-dbf8-4f91-a292-cd6b1d8bc5d1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.108105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 809/1606] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:34.108940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa77325-599f-4dac-b6fe-eb7522602844 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.120448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa77325-599f-4dac-b6fe-eb7522602844 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.125056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748f315-c907-46ec-8fb3-2b7f542db386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.125934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748f315-c907-46ec-8fb3-2b7f542db386 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.126241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748f315-c907-46ec-8fb3-2b7f542db386 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.127222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748f315-c907-46ec-8fb3-2b7f542db386 admin admin] PATH_INFO: `/v3/projects/5f7ed2c773bd43c7a0845599bfb902ec/users/bf1b5d8cbfbb450da8878719c4aa6712/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.141259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748f315-c907-46ec-8fb3-2b7f542db386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7ed2c773bd43c7a0845599bfb902ec, user_id=bf1b5d8cbfbb450da8878719c4aa6712, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.144035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748f315-c907-46ec-8fb3-2b7f542db386 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.306202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 798/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:33 2022] PUT /identity/v3/projects/c100074063c44d96adf93e0b0f080527/users/e367e391e68a477d861a515f846a8dfa/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:34.311506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f55951-b38d-41f0-a794-b7185a4f8ac9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.311865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f55951-b38d-41f0-a794-b7185a4f8ac9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.312204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f55951-b38d-41f0-a794-b7185a4f8ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.331618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-14f55951-b38d-41f0-a794-b7185a4f8ac9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:34.332899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-14f55951-b38d-41f0-a794-b7185a4f8ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:34.384173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 810/1608] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:34 2022] PUT /identity/v3/projects/5f7ed2c773bd43c7a0845599bfb902ec/users/bf1b5d8cbfbb450da8878719c4aa6712/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:34.384173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee70d8-cdea-4c49-8347-e1c2d5574697 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.384173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee70d8-cdea-4c49-8347-e1c2d5574697 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.384173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee70d8-cdea-4c49-8347-e1c2d5574697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.414670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 799/1609] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:34.414670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fccbde-5f2b-4475-bad5-ea4e9a171eee None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.414670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fccbde-5f2b-4475-bad5-ea4e9a171eee None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.414670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fccbde-5f2b-4475-bad5-ea4e9a171eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.416235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-71ee70d8-cdea-4c49-8347-e1c2d5574697 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:34.417601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-71ee70d8-cdea-4c49-8347-e1c2d5574697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:34.442938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c8fccbde-5f2b-4475-bad5-ea4e9a171eee None None] MFA Rules not processed for user `082433ca0c3947efa367a562cb2802c1`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:34.448357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c8fccbde-5f2b-4475-bad5-ea4e9a171eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:34.478790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 811/1610] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:34.479931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee70d8-cdea-4c49-8347-e1c2d5574697 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.490208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee70d8-cdea-4c49-8347-e1c2d5574697 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.494881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.495544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.495834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.496108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.496803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.498774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.514449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 800/1611] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:34.518602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dfaea-2027-477e-9a01-60c197886224 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.518602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dfaea-2027-477e-9a01-60c197886224 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.518602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dfaea-2027-477e-9a01-60c197886224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.535029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8d6dfaea-2027-477e-9a01-60c197886224 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:34.536231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8d6dfaea-2027-477e-9a01-60c197886224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:34.543854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 801/1612] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:34.545046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99879fe2-ba25-426b-82c3-eb9b1c13231d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.545329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99879fe2-ba25-426b-82c3-eb9b1c13231d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.545603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99879fe2-ba25-426b-82c3-eb9b1c13231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.559252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:34.565468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-99879fe2-ba25-426b-82c3-eb9b1c13231d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:34.576506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-99879fe2-ba25-426b-82c3-eb9b1c13231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:34.576506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 802/1613] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:34.576506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99879fe2-ba25-426b-82c3-eb9b1c13231d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.635248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99879fe2-ba25-426b-82c3-eb9b1c13231d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.638933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.639653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.640076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.640397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.649957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.651565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.661169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 803/1614] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:34.662063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.670092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576616b-2cd0-4867-aa3e-90493ddeaf7b service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.674769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.675694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.675845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.676175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.686635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.688134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.699118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 804/1615] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:34.699953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.710261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecca6f4-bb38-42fa-830b-e1f4ffa9f17d service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.716512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.717504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.717801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.718074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.718793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.721256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.754142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 812/1616] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/projects => generated 371 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:34.754142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.807895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dabaf-f333-42f2-ab2f-a8440eb75383 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.811994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.812729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.813040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.813318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.832832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:34.875519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.877698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.888151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 813/1617] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:34.888933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.897783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2667-4774-4b33-b810-1f062743070c service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.902837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.903503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.914643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.914643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.915046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.916542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.928964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 814/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:34.930014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:34.939845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52078727-8ffc-406f-ace1-2a0cd01e290b service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:34.944819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:34.945483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:34.945764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:34.946043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:34.946785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:34.948755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:34.961441 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 805/1619] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/projects => generated 399 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:34.962450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.037475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce445335-e2c6-4a94-a4f3-4ef18b37471f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.042901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.043635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.043905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.044179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.045153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.047296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.130381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 815/1620] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:35.134192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.145047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb679a9-5082-4fc3-8ef4-a1a8f69152a0 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.150145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.150988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.151379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.151777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.152936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.154532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.166131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 816/1621] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:55:35.170344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.183683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0129b-d378-4534-abae-81d82ac2c49a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.190617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.191668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.200140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.200140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] PATH_INFO: `/v3/projects/8f6b9ad34cbe4993aa929df84b7968e8/users/4c1e46c391414137bc4de6aea166dfc1/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.212802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6b9ad34cbe4993aa929df84b7968e8, user_id=4c1e46c391414137bc4de6aea166dfc1, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.216512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.322147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 806/1622] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:34 2022] POST /identity/v3/users => generated 306 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:35.325998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.337829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eb86f-4091-4f49-b6b3-3da807a1e039 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.342569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.343241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.343520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.343795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.344519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.346793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.357395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 807/1623] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:55:35.362696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.372840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82163e-4088-4d01-8a52-3569f8fedda6 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.385838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.386784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.387072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.387350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] PATH_INFO: `/v3/projects/7fcb54c67ac3436e9babfb00b406d3ad/users/65b1c243356146889951c4907d9a0f10/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.404574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fcb54c67ac3436e9babfb00b406d3ad, user_id=65b1c243356146889951c4907d9a0f10, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.407672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.436844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 817/1624] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:35 2022] PUT /identity/v3/projects/8f6b9ad34cbe4993aa929df84b7968e8/users/4c1e46c391414137bc4de6aea166dfc1/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:35.441924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.510336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba63d9-5b85-41f4-993a-a08c925f2575 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.515273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.518800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.518800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.518800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.518800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.519173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.530044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 818/1625] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:55:35.535188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.545915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a438df-e8fb-4dc0-a468-9ad2dd107461 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.561498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 808/1626] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:35 2022] PUT /identity/v3/projects/7fcb54c67ac3436e9babfb00b406d3ad/users/65b1c243356146889951c4907d9a0f10/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:55:35.563143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.643215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:35.653915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce4704-dc54-4fb2-b2d8-7859baa6acb0 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.655706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8906271-e7e5-4b04-9e24-f702477a13c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.657192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8906271-e7e5-4b04-9e24-f702477a13c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.657599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8906271-e7e5-4b04-9e24-f702477a13c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.658532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8906271-e7e5-4b04-9e24-f702477a13c0 admin admin] PATH_INFO: `/v3/projects/8f6b9ad34cbe4993aa929df84b7968e8/users/4c1e46c391414137bc4de6aea166dfc1/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.661591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8906271-e7e5-4b04-9e24-f702477a13c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6b9ad34cbe4993aa929df84b7968e8, user_id=4c1e46c391414137bc4de6aea166dfc1, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.665058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8906271-e7e5-4b04-9e24-f702477a13c0 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.668210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.668944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.669264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.669553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.670334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.672127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.688433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 819/1627] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:35 2022] PUT /identity/v3/projects/8f6b9ad34cbe4993aa929df84b7968e8/users/4c1e46c391414137bc4de6aea166dfc1/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:35.695543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ade2-fca5-4f1e-971d-17b30f82e529 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.696535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ade2-fca5-4f1e-971d-17b30f82e529 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.697025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ade2-fca5-4f1e-971d-17b30f82e529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.699042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 809/1628] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:35.700075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:35.713161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd5345-ac28-4c6d-a523-1c78c115a6fa admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:35.771159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a0c9ade2-fca5-4f1e-971d-17b30f82e529 None None] MFA Rules not processed for user `4c1e46c391414137bc4de6aea166dfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:35.771951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a0c9ade2-fca5-4f1e-971d-17b30f82e529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:35.839132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e31fd-3f55-4736-ac08-b2c7313c8b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:35.844508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e31fd-3f55-4736-ac08-b2c7313c8b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.844508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e31fd-3f55-4736-ac08-b2c7313c8b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.844508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e31fd-3f55-4736-ac08-b2c7313c8b1f admin admin] PATH_INFO: `/v3/projects/7fcb54c67ac3436e9babfb00b406d3ad/users/65b1c243356146889951c4907d9a0f10/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.848976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e31fd-3f55-4736-ac08-b2c7313c8b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fcb54c67ac3436e9babfb00b406d3ad, user_id=65b1c243356146889951c4907d9a0f10, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:35.861886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e31fd-3f55-4736-ac08-b2c7313c8b1f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:35.873285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:35.903146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 810/1629] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:35 2022] PUT /identity/v3/projects/7fcb54c67ac3436e9babfb00b406d3ad/users/65b1c243356146889951c4907d9a0f10/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:35.904843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef72f0-cb73-42a8-858a-f6aa18d5aa81 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.905141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef72f0-cb73-42a8-858a-f6aa18d5aa81 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.905437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef72f0-cb73-42a8-858a-f6aa18d5aa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.923112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 820/1630] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:35.931097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38f1f7-34e3-4d3c-8881-f7b2fe2de3c5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:35.936473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38f1f7-34e3-4d3c-8881-f7b2fe2de3c5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:35.936473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38f1f7-34e3-4d3c-8881-f7b2fe2de3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:35.964316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3fef72f0-cb73-42a8-858a-f6aa18d5aa81 None None] MFA Rules not processed for user `5fdf869a706a445d950ee8f59595ad4a`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:35.967472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3fef72f0-cb73-42a8-858a-f6aa18d5aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:35.990923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6c38f1f7-34e3-4d3c-8881-f7b2fe2de3c5 None None] MFA Rules not processed for user `65b1c243356146889951c4907d9a0f10`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:35.992346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6c38f1f7-34e3-4d3c-8881-f7b2fe2de3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:36.091759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 811/1631] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:36.092629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef72f0-cb73-42a8-858a-f6aa18d5aa81 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:36.138077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 821/1632] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:36.153878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d8d4b-ca90-410d-b3a8-91d4e3e6c4c0 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.154792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d8d4b-ca90-410d-b3a8-91d4e3e6c4c0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.155819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d8d4b-ca90-410d-b3a8-91d4e3e6c4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.183658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-aa3d8d4b-ca90-410d-b3a8-91d4e3e6c4c0 None None] MFA Rules not processed for user `5fdf869a706a445d950ee8f59595ad4a`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:36.185301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-aa3d8d4b-ca90-410d-b3a8-91d4e3e6c4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:36.187652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef72f0-cb73-42a8-858a-f6aa18d5aa81 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:36.193324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:36.198672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 822/1633] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:36.198672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d8d4b-ca90-410d-b3a8-91d4e3e6c4c0 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:36.199608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.199608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.202215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.214520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d8d4b-ca90-410d-b3a8-91d4e3e6c4c0 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:36.218629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e623f-0ffd-4a39-842a-01849c0fe838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:36.219379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e623f-0ffd-4a39-842a-01849c0fe838 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.220012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e623f-0ffd-4a39-842a-01849c0fe838 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.221094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e623f-0ffd-4a39-842a-01849c0fe838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.232228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e623f-0ffd-4a39-842a-01849c0fe838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:36.235011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e623f-0ffd-4a39-842a-01849c0fe838 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:36.250877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 823/1634] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:36.252848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf0985-88f4-4268-aef7-c93cdad6b163 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.253280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf0985-88f4-4268-aef7-c93cdad6b163 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.253741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf0985-88f4-4268-aef7-c93cdad6b163 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.255789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 824/1635] 10.222.0.23 () {54 vars in 961 bytes} [Wed Jun 15 14:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:55:36.291929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833955-d2fd-46e6-92f1-d982472d1fa2 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.292665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:36.293660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833955-d2fd-46e6-92f1-d982472d1fa2 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.294712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833955-d2fd-46e6-92f1-d982472d1fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.295651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:36.317686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 812/1636] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:36.352499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f8833955-d2fd-46e6-92f1-d982472d1fa2 None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:36.353870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f8833955-d2fd-46e6-92f1-d982472d1fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:36.362967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:36.459072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 825/1637] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Jun 15 14:55:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:55:36.467328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:36.481378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee5ad7-306b-415f-95b3-fd7d22f4a703 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:36.490115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:36.490655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.491142 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.491256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.502526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:36.509164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:36.519847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 813/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:36.623030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833955-d2fd-46e6-92f1-d982472d1fa2 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:36.635097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833955-d2fd-46e6-92f1-d982472d1fa2 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:36.640626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7c3c-3f7e-4f5f-a53a-5a83f49a26e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:36.642007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7c3c-3f7e-4f5f-a53a-5a83f49a26e3 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.642453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7c3c-3f7e-4f5f-a53a-5a83f49a26e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.643117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7c3c-3f7e-4f5f-a53a-5a83f49a26e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.771512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af7c3c-3f7e-4f5f-a53a-5a83f49a26e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:36.774148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af7c3c-3f7e-4f5f-a53a-5a83f49a26e3 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:36.796411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 826/1639] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:36.808313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:36.823912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770afdde-fb61-4837-b0c9-549039b3b5ab service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:36.846447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc03a5-417a-47cd-937d-fc643678795d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:36.847210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc03a5-417a-47cd-937d-fc643678795d service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.849565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc03a5-417a-47cd-937d-fc643678795d service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.849859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc03a5-417a-47cd-937d-fc643678795d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.883884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f830-6a7d-4324-b262-765336f18e44 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.884172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f830-6a7d-4324-b262-765336f18e44 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.884456 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f830-6a7d-4324-b262-765336f18e44 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.886081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 827/1640] 10.222.0.23 () {54 vars in 912 bytes} [Wed Jun 15 14:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:55:36.892924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:36.893163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:36.893438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:36.928900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:36.932279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:36.956124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc03a5-417a-47cd-937d-fc643678795d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:36.958124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc03a5-417a-47cd-937d-fc643678795d service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:36.958595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 828/1641] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:55:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:55:36.973562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:36.975005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:36.988078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 814/1642] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:37.089672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853db0d-f2d2-40e5-bc8a-4abd4331fae6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:37.099087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a37b8-c5d8-4423-9b70-b927dda45631 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:37.099756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a37b8-c5d8-4423-9b70-b927dda45631 service glance] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:37.100044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a37b8-c5d8-4423-9b70-b927dda45631 service glance] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:37.100321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a37b8-c5d8-4423-9b70-b927dda45631 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:37.123948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a37b8-c5d8-4423-9b70-b927dda45631 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:37.126027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a37b8-c5d8-4423-9b70-b927dda45631 service glance] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:37.167412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 829/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:55:37.196740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:37.645327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cbb12-f901-4a2c-8f05-0b96bcdb9bdf None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:37.645451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cbb12-f901-4a2c-8f05-0b96bcdb9bdf None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:37.646107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cbb12-f901-4a2c-8f05-0b96bcdb9bdf None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:37.647723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 815/1644] 10.222.0.23 () {54 vars in 912 bytes} [Wed Jun 15 14:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:55:37.657240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec186443-81fa-469f-8e8f-c75b9f473ead None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:37.657571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec186443-81fa-469f-8e8f-c75b9f473ead None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:37.657848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec186443-81fa-469f-8e8f-c75b9f473ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:37.731341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ec186443-81fa-469f-8e8f-c75b9f473ead None None] MFA Rules not processed for user `bead604b09be482c906a97b53bb03d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:37.733336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ec186443-81fa-469f-8e8f-c75b9f473ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:37.815813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 830/1645] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:55:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:37.831336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cbb12-f901-4a2c-8f05-0b96bcdb9bdf None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:37.843701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cbb12-f901-4a2c-8f05-0b96bcdb9bdf None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:37.847510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:37.848188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:37.848450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:37.848723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:37.859854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:37.861965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:37.871701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 816/1646] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:38.284444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f3e43-2d43-4f6f-aecc-d1783e2adce7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:38.284444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f3e43-2d43-4f6f-aecc-d1783e2adce7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:38.284444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f3e43-2d43-4f6f-aecc-d1783e2adce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:38.284444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e15f3e43-2d43-4f6f-aecc-d1783e2adce7 None None] MFA Rules not processed for user `e0d94fbf267b4152920e0fd30c929740`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:38.284444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e15f3e43-2d43-4f6f-aecc-d1783e2adce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:38.284444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 831/1647] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:37 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:38.295914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:38.326576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e780e59-2f1e-45be-851e-c2fddc126a04 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:38.341252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d735bbc-e1aa-4a06-9c9a-554e5f2f5143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:38.343069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735bbc-e1aa-4a06-9c9a-554e5f2f5143 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:38.343194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735bbc-e1aa-4a06-9c9a-554e5f2f5143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:38.343458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735bbc-e1aa-4a06-9c9a-554e5f2f5143 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:38.346486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735bbc-e1aa-4a06-9c9a-554e5f2f5143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:38.351388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735bbc-e1aa-4a06-9c9a-554e5f2f5143 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:38.566401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 817/1648] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:38 2022] POST /identity/v3/projects => generated 403 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:38.577788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f3e43-2d43-4f6f-aecc-d1783e2adce7 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:38.584062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f3e43-2d43-4f6f-aecc-d1783e2adce7 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:38.694589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:38.695525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:38.695854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:38.696397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:38.697032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:38.699515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:38.958855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480c376-8187-442b-be99-b9d9d756ed36 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:38.959418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480c376-8187-442b-be99-b9d9d756ed36 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:38.959418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480c376-8187-442b-be99-b9d9d756ed36 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:38.961814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 818/1649] 10.222.0.23 () {54 vars in 960 bytes} [Wed Jun 15 14:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:55:38.968556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34598c0f-e065-48ac-8da3-57a6bd92d2d5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:38.970823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34598c0f-e065-48ac-8da3-57a6bd92d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:38.970823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34598c0f-e065-48ac-8da3-57a6bd92d2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.007960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-34598c0f-e065-48ac-8da3-57a6bd92d2d5 None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:39.009234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-34598c0f-e065-48ac-8da3-57a6bd92d2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:39.024094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 832/1650] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:38 2022] POST /identity/v3/users => generated 308 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:55:39.025593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:39.046713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77006cd2-5545-4fd6-abe5-b46b1fc4a117 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:39.058740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:39.059693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.060126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.060548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.061440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:39.064911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:39.101414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 833/1651] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:39.102329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:39.121943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e0b39-91fb-43bb-ab0c-4d82a86a7fc6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:39.140349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e3921-f41e-4f3c-82f2-e2f132e1c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:39.141277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3921-f41e-4f3c-82f2-e2f132e1c18e admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.141562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3921-f41e-4f3c-82f2-e2f132e1c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.141836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3921-f41e-4f3c-82f2-e2f132e1c18e admin admin] PATH_INFO: `/v3/projects/33250190bfe245ba9c6c50f903892222/users/d7418c074c544b35ae41958d27bf2ef9/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.156467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 819/1652] 10.222.0.23 () {58 vars in 1057 bytes} [Wed Jun 15 14:55:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:39.168182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34598c0f-e065-48ac-8da3-57a6bd92d2d5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:39.179034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34598c0f-e065-48ac-8da3-57a6bd92d2d5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:39.197228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:39.197910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.198232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.198512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.199863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e3921-f41e-4f3c-82f2-e2f132e1c18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33250190bfe245ba9c6c50f903892222, user_id=d7418c074c544b35ae41958d27bf2ef9, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:39.205253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:39.206592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e3921-f41e-4f3c-82f2-e2f132e1c18e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:39.231379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:39.235033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:39.258843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 820/1653] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:55:39.358310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 834/1654] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:39 2022] PUT /identity/v3/projects/33250190bfe245ba9c6c50f903892222/users/d7418c074c544b35ae41958d27bf2ef9/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:39.368992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:39.392128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a3a10-d7f2-45b0-b37a-0c5beb5a97fc service placement] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:39.534440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd33583-9748-4f52-a144-8f64735d2d96 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.534738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd33583-9748-4f52-a144-8f64735d2d96 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.535018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd33583-9748-4f52-a144-8f64735d2d96 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.536734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 835/1655] 10.222.0.23 () {54 vars in 961 bytes} [Wed Jun 15 14:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:55:39.547328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b4483-230f-4be0-ba17-585965518e2a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.556318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b4483-230f-4be0-ba17-585965518e2a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.556757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b4483-230f-4be0-ba17-585965518e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.607126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9d6b4483-230f-4be0-ba17-585965518e2a None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:39.607126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9d6b4483-230f-4be0-ba17-585965518e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:39.639293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:39.641050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.641050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.641050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.643501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:39.645383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:39.673594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 821/1656] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:39.675063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:39.697308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a972753-1763-425c-80b7-6a49255eb326 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:39.715584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 836/1657] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Jun 15 14:55:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 15 14:55:39.717397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:39.718349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.718628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.718905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] PATH_INFO: `/v3/projects/33250190bfe245ba9c6c50f903892222/users/d7418c074c544b35ae41958d27bf2ef9/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.720626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412330c-8802-47ee-b57b-34980129ea49 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.721452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412330c-8802-47ee-b57b-34980129ea49 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.721729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412330c-8802-47ee-b57b-34980129ea49 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.723443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 837/1658] 10.222.0.23 () {54 vars in 912 bytes} [Wed Jun 15 14:55:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:55:39.729910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca29310-6626-4bb5-8889-753ebf5778bc None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:39.730254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca29310-6626-4bb5-8889-753ebf5778bc None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:39.730496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca29310-6626-4bb5-8889-753ebf5778bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:39.732861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33250190bfe245ba9c6c50f903892222, user_id=d7418c074c544b35ae41958d27bf2ef9, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:39.736676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:39.752577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:39.787520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 822/1659] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:55:39 2022] PUT /identity/v3/projects/33250190bfe245ba9c6c50f903892222/users/d7418c074c544b35ae41958d27bf2ef9/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:39.810552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1ca29310-6626-4bb5-8889-753ebf5778bc None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:39.812469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1ca29310-6626-4bb5-8889-753ebf5778bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:39.922619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 838/1660] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:55:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:39.936621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:40.032467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed63d6-2034-463a-a13e-723025f87ad4 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:40.041021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96aa819-3520-46ac-998c-bcf36dbfa4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:40.041021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96aa819-3520-46ac-998c-bcf36dbfa4ac service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:40.041021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96aa819-3520-46ac-998c-bcf36dbfa4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:40.041021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96aa819-3520-46ac-998c-bcf36dbfa4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:40.060544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96aa819-3520-46ac-998c-bcf36dbfa4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:40.062541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96aa819-3520-46ac-998c-bcf36dbfa4ac service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:40.077224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 823/1661] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:40.763972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:41.539917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:42.418512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:43.379258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e21cc-3820-4b33-8422-75f622851a3b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:43.379614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e21cc-3820-4b33-8422-75f622851a3b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:43.379842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e21cc-3820-4b33-8422-75f622851a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:43.449594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-8a2e21cc-3820-4b33-8422-75f622851a3b None None] MFA Rules not processed for user `d7418c074c544b35ae41958d27bf2ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:43.451514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8a2e21cc-3820-4b33-8422-75f622851a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:43.636465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 839/1662] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:55:45.988142 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41c11-5a0b-4a84-af8d-3a1c4ce6b39e None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:45.989137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41c11-5a0b-4a84-af8d-3a1c4ce6b39e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:45.992543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41c11-5a0b-4a84-af8d-3a1c4ce6b39e None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:46.002658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 824/1663] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:55:45 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:55:46.142985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c841b-aa75-47cd-b1a3-883f38cb062a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:46.142985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c841b-aa75-47cd-b1a3-883f38cb062a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:46.142985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c841b-aa75-47cd-b1a3-883f38cb062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:46.207527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-845c841b-aa75-47cd-b1a3-883f38cb062a None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:46.209181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-845c841b-aa75-47cd-b1a3-883f38cb062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:46.332898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:46.382050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 840/1664] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:55:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:46.527955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d41c11-5a0b-4a84-af8d-3a1c4ce6b39e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:46.557687 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d41c11-5a0b-4a84-af8d-3a1c4ce6b39e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:46.566434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:46.567457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:46.567996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:46.568479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:46.584321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:46.587491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:46.614743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 825/1665] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:48.131970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:48.132414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:48.132543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:48.193989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:48.196906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:48.437866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 841/1666] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:55:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:55:49.002996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:49.100188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433ee79-6588-4c73-a6f5-2174f7d35b17 service placement] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:49.107118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:49.108329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:49.109352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:49.109796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] PATH_INFO: `/v3/users/082433ca0c3947efa367a562cb2802c1` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:49.112877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=082433ca0c3947efa367a562cb2802c1)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:49.114229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: WARNING py.warnings [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:55:49.114229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: warnings.warn(msg) Jun 15 14:55:49.114229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: Jun 15 14:55:49.116513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:49.167639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:49.169211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:49.346955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641e43a-28d2-41f5-93f0-1f0640322d5d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:49.353198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce8c95-2ab5-4b81-b71e-5fca59c98f66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:49.354111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8c95-2ab5-4b81-b71e-5fca59c98f66 service swift] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:49.354553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8c95-2ab5-4b81-b71e-5fca59c98f66 service swift] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:49.355462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8c95-2ab5-4b81-b71e-5fca59c98f66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:49.411855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce8c95-2ab5-4b81-b71e-5fca59c98f66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:49.415123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce8c95-2ab5-4b81-b71e-5fca59c98f66 service swift] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:49.437006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082433ca0c3947efa367a562cb2802c1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:49.442211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082433ca0c3947efa367a562cb2802c1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:49.458587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 842/1667] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:55:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:55:49.467319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:49.469211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:49.469639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:49.483225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082433ca0c3947efa367a562cb2802c1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:49.571044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] MFA Rules not processed for user `bf1b5d8cbfbb450da8878719c4aa6712`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:49.573483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:49.614061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082433ca0c3947efa367a562cb2802c1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:49.684573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a6f9b5d9-1e53-4b9e-a99f-5dc0fbacc17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082433ca0c3947efa367a562cb2802c1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:49.715268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 826/1668] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:55:49 2022] DELETE /identity/v3/users/082433ca0c3947efa367a562cb2802c1 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:49.734523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:49.736358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:49.736906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:49.773356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 843/1669] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:49.794582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:49.797286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:49.851480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:49.853081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:49.882940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3260e-8a4f-4ea3-9388-afeb28025c0c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:49.896983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:49.968674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:50.121767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 827/1670] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:50.206800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:50.208767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:50.210589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:50.211109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:50.408396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:50.411002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:50.446245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 844/1671] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:55:50.502077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:50.503787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:50.779694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630693f-1ade-4921-8d45-fc46ae028a0b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:50.779694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12e47d-2891-4cd2-8912-924083197471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:50.779694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12e47d-2891-4cd2-8912-924083197471 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:50.779694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12e47d-2891-4cd2-8912-924083197471 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:50.780256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12e47d-2891-4cd2-8912-924083197471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:50.780256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12e47d-2891-4cd2-8912-924083197471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:50.780256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12e47d-2891-4cd2-8912-924083197471 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:50.818509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 828/1672] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:51.176598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:51.178958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:51.209183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66688bd9-6335-4f95-8c89-ec9e7577f014 service nova] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:51.235139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:51.237939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:51.239825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:51.240274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] PATH_INFO: `/v3/users/e0d94fbf267b4152920e0fd30c929740` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:51.243977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d94fbf267b4152920e0fd30c929740)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:51.253581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:51.554893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d94fbf267b4152920e0fd30c929740'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:51.557131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d94fbf267b4152920e0fd30c929740'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:51.560133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d94fbf267b4152920e0fd30c929740'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:51.599346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d94fbf267b4152920e0fd30c929740'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:51.739455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-454dc46d-16be-42cb-9b09-a0b25f48656c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d94fbf267b4152920e0fd30c929740'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:51.781818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 845/1673] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:55:51 2022] DELETE /identity/v3/users/e0d94fbf267b4152920e0fd30c929740 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:52.297598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:52.299740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:52.300170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:52.390130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] MFA Rules not processed for user `bead604b09be482c906a97b53bb03d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:52.393783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:52.542034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:52.542034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:52.542034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:52.705308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] MFA Rules not processed for user `df400aee39a3498d8a26d55180075ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:52.705308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:52.705603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 829/1674] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Jun 15 14:55:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:52.710753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:52.717463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:52.720002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:52.766743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134af31-92f8-4769-a46c-4c5e60ec8a4c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:52.792002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:52.796612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:52.798074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:52.800778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:52.837487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:52.844354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:52.920912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 830/1675] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:55:52.923411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 846/1676] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:52.930368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:52.931832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:53.131656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da639cad-9654-45fa-811b-6d8c03b6278a service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:53.150919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:53.151968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:53.152430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:53.152730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:53.176802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:53.183623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:53.214858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 831/1677] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:53.263936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:53.264111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:53.305152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:53.309262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:53.452127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f81ccc-1c69-44b3-b893-a5f59dbf8743 service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:53.457204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:53.459285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:53.459940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:53.460812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] PATH_INFO: `/v3/projects/afc5256083884a55982634bb457f7dba` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:53.463303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc5256083884a55982634bb457f7dba)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:53.467667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:53.580872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ae279-5168-4172-a8b0-18e124e3d9c8 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:53.594781 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:53.598479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:53.598983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:53.599413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] PATH_INFO: `/v3/projects/44a5d9a7cb35407f8accbaff750ba980` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:53.601403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a5d9a7cb35407f8accbaff750ba980)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:53.604853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: WARNING py.warnings [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 14:55:53.604853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: warnings.warn(msg) Jun 15 14:55:53.604853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: Jun 15 14:55:53.609010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:53.820163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc5256083884a55982634bb457f7dba', 'request_id': 'req-50583bda-d50c-404c-9069-33c0945aaa79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:53.847744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc5256083884a55982634bb457f7dba', 'request_id': 'req-50583bda-d50c-404c-9069-33c0945aaa79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:53.942510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a5d9a7cb35407f8accbaff750ba980', 'request_id': 'req-9bc74744-5c20-4945-b217-d1faaebfa494'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:53.998531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc5256083884a55982634bb457f7dba', 'request_id': 'req-50583bda-d50c-404c-9069-33c0945aaa79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:54.020675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] The token cache is being invalidate because project afc5256083884a55982634bb457f7dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:55:54.022237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:54.025330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:54.029103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 832/1678] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:55:53 2022] DELETE /identity/v3/projects/afc5256083884a55982634bb457f7dba => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:54.033017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:54.034713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:54.238952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583bda-d50c-404c-9069-33c0945aaa79 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:54.425849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:54.427641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:54.430418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:54.430766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] PATH_INFO: `/v3/users/d7418c074c544b35ae41958d27bf2ef9` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:54.433267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7418c074c544b35ae41958d27bf2ef9)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:54.435782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:54.582770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a5d9a7cb35407f8accbaff750ba980', 'request_id': 'req-9bc74744-5c20-4945-b217-d1faaebfa494'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:54.600733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a5d9a7cb35407f8accbaff750ba980', 'request_id': 'req-9bc74744-5c20-4945-b217-d1faaebfa494'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:54.667075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] The token cache is being invalidate because project 44a5d9a7cb35407f8accbaff750ba980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:55:54.669438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:54.672139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:54.680363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 847/1679] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:55:53 2022] DELETE /identity/v3/projects/44a5d9a7cb35407f8accbaff750ba980 => generated 0 bytes in 1414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:54.681467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:54.707139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:54.746243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc74744-5c20-4945-b217-d1faaebfa494 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:54.773748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:54.784539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:54.796224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:54.796224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] PATH_INFO: `/v3/users/65b1c243356146889951c4907d9a0f10` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:54.796224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b1c243356146889951c4907d9a0f10)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:54.808416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:55.237494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7418c074c544b35ae41958d27bf2ef9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.241213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7418c074c544b35ae41958d27bf2ef9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.282406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7418c074c544b35ae41958d27bf2ef9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.391645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7418c074c544b35ae41958d27bf2ef9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.397505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b1c243356146889951c4907d9a0f10'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.403439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b1c243356146889951c4907d9a0f10'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.403439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b1c243356146889951c4907d9a0f10'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.454593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b1c243356146889951c4907d9a0f10'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.471006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d9d0d76f-15e6-49f5-85b3-24041363affa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7418c074c544b35ae41958d27bf2ef9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.495890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 833/1680] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:55:54 2022] DELETE /identity/v3/users/d7418c074c544b35ae41958d27bf2ef9 => generated 0 bytes in 1463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:55.578319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:55.578865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:55.580864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:55.674713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] MFA Rules not processed for user `cbc9d5ce4a3241ea898f216543114fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:55.678907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:55.866273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e4f1575-3ffd-45e6-af39-74b727e33e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b1c243356146889951c4907d9a0f10'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:55.917039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 848/1681] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:55:54 2022] DELETE /identity/v3/users/65b1c243356146889951c4907d9a0f10 => generated 0 bytes in 1232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:55.980313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d447d9-01b2-4e3d-8509-0dd3a2041281 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:55.980791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d447d9-01b2-4e3d-8509-0dd3a2041281 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:55.981418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d447d9-01b2-4e3d-8509-0dd3a2041281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:55.990528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 834/1682] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:55:56.009554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-30d447d9-01b2-4e3d-8509-0dd3a2041281 None None] MFA Rules not processed for user `cbc9d5ce4a3241ea898f216543114fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:56.012303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-30d447d9-01b2-4e3d-8509-0dd3a2041281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:56.165544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 849/1683] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:56.172119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:56.176535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:56.279125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e1ce9-0f46-4cc4-b666-3bdce48094a5 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:56.280206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e1ce9-0f46-4cc4-b666-3bdce48094a5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:56.280859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e1ce9-0f46-4cc4-b666-3bdce48094a5 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:56.282796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 850/1684] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:55:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:55:56.294091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79534f66-b101-40e8-a8d0-d6d05fa54b5e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:56.317617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:56.341773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:56.341773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:56.368483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:56.370546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:56.464067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:56.465353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:56.465794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:56.466303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:56.479555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:56.482855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:56.500075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:55:56.511298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 835/1685] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:55:56.599496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 851/1686] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:57.502489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:57.502489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:57.662532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:57.664062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:57.695966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f0766-cc22-4f49-a166-6365fc5bffc4 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:57.706518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0ca47-c1f5-4ea4-9160-53d23dd44268 service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:57.718728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7009-35c6-480e-b8cd-9c53fffc74fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:57.719410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7009-35c6-480e-b8cd-9c53fffc74fd service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:57.719693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7009-35c6-480e-b8cd-9c53fffc74fd service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:57.719972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7009-35c6-480e-b8cd-9c53fffc74fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:57.727138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:57.728324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:57.734660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:57.734968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] PATH_INFO: `/v3/projects/33250190bfe245ba9c6c50f903892222` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:57.739229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33250190bfe245ba9c6c50f903892222)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:57.744646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:57.761739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec7009-35c6-480e-b8cd-9c53fffc74fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:57.764561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec7009-35c6-480e-b8cd-9c53fffc74fd service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:57.824589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 852/1687] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:55:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:55:57.995154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33250190bfe245ba9c6c50f903892222', 'request_id': 'req-66af9ce5-98d5-4f19-88ef-444557a7bdc1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:58.006745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33250190bfe245ba9c6c50f903892222', 'request_id': 'req-66af9ce5-98d5-4f19-88ef-444557a7bdc1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:58.374346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33250190bfe245ba9c6c50f903892222', 'request_id': 'req-66af9ce5-98d5-4f19-88ef-444557a7bdc1'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:58.403595 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] The token cache is being invalidate because project 33250190bfe245ba9c6c50f903892222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:55:58.403770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:58.411745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-66af9ce5-98d5-4f19-88ef-444557a7bdc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:55:58.416452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 836/1688] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:55:57 2022] DELETE /identity/v3/projects/33250190bfe245ba9c6c50f903892222 => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:55:58.421220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ce7ee-ff16-44ff-a061-51c0d196bc7c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:58.421520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ce7ee-ff16-44ff-a061-51c0d196bc7c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:58.421801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ce7ee-ff16-44ff-a061-51c0d196bc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:58.461875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-eb2ce7ee-ff16-44ff-a061-51c0d196bc7c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:58.471313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-eb2ce7ee-ff16-44ff-a061-51c0d196bc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:58.627456 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 853/1689] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:58.635025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f40c-b045-4fe9-b57a-0767365d9967 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:58.636061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f40c-b045-4fe9-b57a-0767365d9967 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:58.639105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f40c-b045-4fe9-b57a-0767365d9967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:58.706094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1d35f40c-b045-4fe9-b57a-0767365d9967 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:55:58.708023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1d35f40c-b045-4fe9-b57a-0767365d9967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:58.743584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ce7ee-ff16-44ff-a061-51c0d196bc7c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:58.750085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 837/1690] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:55:58.755938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ce7ee-ff16-44ff-a061-51c0d196bc7c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:58.773275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:58.782131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:58.782131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:58.782131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:58.782131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:55:58.782131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:58.782131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:58.834745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 854/1691] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:55:58.838995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f40c-b045-4fe9-b57a-0767365d9967 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:58.849911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f40c-b045-4fe9-b57a-0767365d9967 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:58.867460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:58.868435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:58.868876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:58.869320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:58.872210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:58.876251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:59.135713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 838/1692] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:55:58 2022] POST /identity/v3/projects => generated 363 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:55:59.136901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:59.159782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4656e-0601-4533-8488-59c7d6010854 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:59.509835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:59.509835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:59.509835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:59.509835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:59.513070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:59.513070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:59.774112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:59.781125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 855/1693] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:55:59 2022] POST /identity/v3/users => generated 282 bytes in 643 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:55:59.797217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef6404-b574-4f60-9fae-5b9601bd8a37 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:59.805209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:59.806863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:59.810478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:59.817772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:59.818843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:59.820730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:59.834566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:59.836762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:55:59.867648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b303a1e-d3f8-40f2-ab35-90b208ae656e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:59.890833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 839/1694] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:55:59.890833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:55:59.897334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:59.898257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:59.906643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:59.906925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] PATH_INFO: `/v3/projects/7fcb54c67ac3436e9babfb00b406d3ad` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:59.910008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fcb54c67ac3436e9babfb00b406d3ad)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:59.912252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:55:59.932537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5efc8-316a-4a4b-8be2-e66240578390 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:55:59.965927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:55:59.966671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:55:59.966671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:55:59.967222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:55:59.967971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:55:59.984679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:00.385909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:00.385909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 840/1695] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:00.385909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:00.385909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d2721-1428-4b0c-a03b-1583196b856f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:00.389094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b671fc-2365-4493-b7dd-edb40c036a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:00.389094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b671fc-2365-4493-b7dd-edb40c036a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.389094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b671fc-2365-4493-b7dd-edb40c036a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.389094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b671fc-2365-4493-b7dd-edb40c036a80 admin admin] PATH_INFO: `/v3/projects/c8323dd57f014e3b911a078f40cbda89/users/680e497764614a21b0182954d82bd0f9/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.391262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b671fc-2365-4493-b7dd-edb40c036a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8323dd57f014e3b911a078f40cbda89, user_id=680e497764614a21b0182954d82bd0f9, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:00.391262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b671fc-2365-4493-b7dd-edb40c036a80 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:00.668112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 841/1696] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:00 2022] PUT /identity/v3/projects/c8323dd57f014e3b911a078f40cbda89/users/680e497764614a21b0182954d82bd0f9/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:00.676063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de10d8-fd19-4729-b750-fe7cac8b4768 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.676390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de10d8-fd19-4729-b750-fe7cac8b4768 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.677161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de10d8-fd19-4729-b750-fe7cac8b4768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:00.749344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fcb54c67ac3436e9babfb00b406d3ad', 'request_id': 'req-231cd671-9011-4bcc-a25b-00de6472c053'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.848877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-70de10d8-fd19-4729-b750-fe7cac8b4768 None None] MFA Rules not processed for user `e367e391e68a477d861a515f846a8dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:00.850161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-70de10d8-fd19-4729-b750-fe7cac8b4768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:00.871701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fcb54c67ac3436e9babfb00b406d3ad', 'request_id': 'req-231cd671-9011-4bcc-a25b-00de6472c053'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.879520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fcb54c67ac3436e9babfb00b406d3ad', 'request_id': 'req-231cd671-9011-4bcc-a25b-00de6472c053'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.939297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] The token cache is being invalidate because project 7fcb54c67ac3436e9babfb00b406d3ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:00.939618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.943449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-231cd671-9011-4bcc-a25b-00de6472c053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:00.952129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 856/1697] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:55:59 2022] DELETE /identity/v3/projects/7fcb54c67ac3436e9babfb00b406d3ad => generated 0 bytes in 1118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:00.956160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f252f7-8cd4-441b-bfff-a2491cd76af0 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:00.957983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f252f7-8cd4-441b-bfff-a2491cd76af0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:00.958302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f252f7-8cd4-441b-bfff-a2491cd76af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.082226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-34f252f7-8cd4-441b-bfff-a2491cd76af0 None None] MFA Rules not processed for user `680e497764614a21b0182954d82bd0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:01.083724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-34f252f7-8cd4-441b-bfff-a2491cd76af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.198699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 842/1698] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:01.200398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.200695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.200973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.250326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:01.251560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.368360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 857/1699] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:01.372145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.372684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.372981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.422553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] MFA Rules not processed for user `cbc9d5ce4a3241ea898f216543114fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:01.424402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.433877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 843/1700] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:01.434629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.447180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.569041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 858/1701] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:01.574484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.583211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:01.604322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67258b-d761-4d63-9f6c-d67b380382a0 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.613533 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.614632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.615300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.615809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.617196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.620321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.714744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728be5a6-979e-4578-b787-dcdcd62482d0 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:01.725166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7294477-d854-426b-ad56-6a8c454d0649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:01.725166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7294477-d854-426b-ad56-6a8c454d0649 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.725166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7294477-d854-426b-ad56-6a8c454d0649 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.725166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7294477-d854-426b-ad56-6a8c454d0649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.733101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7294477-d854-426b-ad56-6a8c454d0649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:01.735208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7294477-d854-426b-ad56-6a8c454d0649 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:01.775721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 844/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:01.778161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebdcc4-3400-446f-94a4-f4c61773d1e2 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:01.778729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebdcc4-3400-446f-94a4-f4c61773d1e2 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:01.779234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebdcc4-3400-446f-94a4-f4c61773d1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:01.797410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 859/1703] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:01.802820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:01.838297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:01.895445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-bdebdcc4-3400-446f-94a4-f4c61773d1e2 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:01.901851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bdebdcc4-3400-446f-94a4-f4c61773d1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:02.241640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 845/1704] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:02.241640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebdcc4-3400-446f-94a4-f4c61773d1e2 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:02.242401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786cd1-8975-4fdd-a757-47fab19d4b2b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.243784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebdcc4-3400-446f-94a4-f4c61773d1e2 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.244059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ccb3e-39b3-4660-afd3-d67ad17e6a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.244059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ccb3e-39b3-4660-afd3-d67ad17e6a21 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.244059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ccb3e-39b3-4660-afd3-d67ad17e6a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.244479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.244677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ccb3e-39b3-4660-afd3-d67ad17e6a21 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.244828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.244828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.244828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.246676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ccb3e-39b3-4660-afd3-d67ad17e6a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.246857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:02.246857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.246857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:02.247282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ccb3e-39b3-4660-afd3-d67ad17e6a21 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:02.247482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 846/1705] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:56:02.253382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:02.290679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6aaa1-8722-4cc9-8fec-1806aafed665 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.297176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.299715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.299715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.299715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.299715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.302758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:02.582664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 860/1706] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:01 2022] POST /identity/v3/users => generated 289 bytes in 780 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:02.586399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664224d-9724-421f-af36-ccb16cad4e37 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.590825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664224d-9724-421f-af36-ccb16cad4e37 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.591323 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664224d-9724-421f-af36-ccb16cad4e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.665960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c664224d-9724-421f-af36-ccb16cad4e37 None None] MFA Rules not processed for user `bead604b09be482c906a97b53bb03d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:02.668272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c664224d-9724-421f-af36-ccb16cad4e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:02.677242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 847/1707] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:02 2022] POST /identity/v3/projects => generated 385 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:02.678268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:02.941133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 861/1708] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:56:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:02.941133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664224d-9724-421f-af36-ccb16cad4e37 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:02.948861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020882d0-62a9-4ed0-ad10-a3700a37efd1 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:02.976166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:02.977235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:02.978766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:02.979090 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:02.980208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:02.983330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:03.003838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 848/1709] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:03.005132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:03.007259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:03.044717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a22a-fd3a-4c05-8a6e-de0ada43e427 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:03.050789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:03.051838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:03.052370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:03.052905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:03.071301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:03.073042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:03.090435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 849/1710] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:03.093796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:03.099414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664224d-9724-421f-af36-ccb16cad4e37 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:03.126060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196de44-b868-4272-be4e-983857bc6898 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:03.129562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:03.130742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:03.131912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:03.132993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:03.135700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:03.138878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:03.139129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:03.139827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:03.140107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:03.140537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] PATH_INFO: `/v3/projects/11fa1155f7634aa5810903ca83953982/users/2d04f40ef60d4416bd775d15d3986084/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:03.183055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fa1155f7634aa5810903ca83953982, user_id=2d04f40ef60d4416bd775d15d3986084, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:03.185875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:03.361830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 862/1711] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:02 2022] POST /identity/v3/users => generated 293 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:03.361830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:03.433316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 850/1712] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:03 2022] PUT /identity/v3/projects/11fa1155f7634aa5810903ca83953982/users/2d04f40ef60d4416bd775d15d3986084/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:03.456019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44cd84-da1e-4c4c-a46f-8c2cc33b4d64 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:03.456214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:03.709721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa0a85-f1dc-44cd-b4bd-d823cbeb98e2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:03.732848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:03.738390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:03.757067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:03.757067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:03.757427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:03.757604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:03.757679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:03.757679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:03.757679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:03.757859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:03.758009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:03.758009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:03.807376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 863/1713] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:03.808541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:03.810593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 851/1714] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:03.812986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:03.836018 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560ca28-690d-4998-9009-746e9c52a992 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:03.842071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe37265-6ca8-4ec7-865e-85419c788ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:03.843293 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe37265-6ca8-4ec7-865e-85419c788ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:03.843924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe37265-6ca8-4ec7-865e-85419c788ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:03.844228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe37265-6ca8-4ec7-865e-85419c788ac3 admin admin] PATH_INFO: `/v3/projects/11fa1155f7634aa5810903ca83953982/users/2d04f40ef60d4416bd775d15d3986084/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:03.847660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe37265-6ca8-4ec7-865e-85419c788ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fa1155f7634aa5810903ca83953982, user_id=2d04f40ef60d4416bd775d15d3986084, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:03.850949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe37265-6ca8-4ec7-865e-85419c788ac3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:03.867993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 864/1715] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:03 2022] PUT /identity/v3/projects/11fa1155f7634aa5810903ca83953982/users/2d04f40ef60d4416bd775d15d3986084/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:03.884134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6298382-dfad-4689-8c68-89094065fd53 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:03.888246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:03.888826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:03.889450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:03.975909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] MFA Rules not processed for user `2d04f40ef60d4416bd775d15d3986084`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:03.978589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:04.060655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:04.061666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:04.062033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:04.062416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:04.063983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:04.067580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:04.146637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 852/1716] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:04.155490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:04.199686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 865/1717] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:04.463903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:04.476383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bedfe-a400-4a85-8725-c1eb6e4e14c3 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:04.476602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:04.477136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:04.477136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:04.477136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:04.477136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] PATH_INFO: `/v3/projects/d8aec72d87bb41c68e1337dcad262434/users/9bb1d24279e64d9e85a3d43a1c59014e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:04.477603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8aec72d87bb41c68e1337dcad262434, user_id=9bb1d24279e64d9e85a3d43a1c59014e, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:04.477603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:04.477761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ef69a-43d8-4200-97ac-be6f399978e9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:04.477761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d7e76-ff7a-4172-a066-bb83358c967b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:04.477761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d7e76-ff7a-4172-a066-bb83358c967b service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:04.477761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d7e76-ff7a-4172-a066-bb83358c967b service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:04.478109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d7e76-ff7a-4172-a066-bb83358c967b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:04.478109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d7e76-ff7a-4172-a066-bb83358c967b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:04.478109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d7e76-ff7a-4172-a066-bb83358c967b service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:04.529100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 866/1718] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:04.606906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 853/1719] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:04 2022] PUT /identity/v3/projects/d8aec72d87bb41c68e1337dcad262434/users/9bb1d24279e64d9e85a3d43a1c59014e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:04.618754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8fb25-5134-4a44-9611-0ce08f642f20 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:04.619255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8fb25-5134-4a44-9611-0ce08f642f20 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:04.620281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8fb25-5134-4a44-9611-0ce08f642f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:04.730626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d5a8fb25-5134-4a44-9611-0ce08f642f20 None None] MFA Rules not processed for user `9bb1d24279e64d9e85a3d43a1c59014e`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:04.737191 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d5a8fb25-5134-4a44-9611-0ce08f642f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:04.993879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 867/1720] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:05.363716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:05.413759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28b02-f99b-4814-930a-5d2669b7810e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:05.414280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28b02-f99b-4814-930a-5d2669b7810e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:05.414711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28b02-f99b-4814-930a-5d2669b7810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:05.447170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-cdf28b02-f99b-4814-930a-5d2669b7810e None None] MFA Rules not processed for user `bf1b5d8cbfbb450da8878719c4aa6712`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:05.447170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cdf28b02-f99b-4814-930a-5d2669b7810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:05.585821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 868/1721] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:05.631112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c04506-2d08-4f09-90f4-c34d05c20a64 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:05.640925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:05.643219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:05.645040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:05.645585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:05.672526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:05.674458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:05.727709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 854/1722] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:06.451253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e175bc-4c0e-49db-9f68-93efec31e49a None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:06.451531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e175bc-4c0e-49db-9f68-93efec31e49a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:06.451997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e175bc-4c0e-49db-9f68-93efec31e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:06.499034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-57e175bc-4c0e-49db-9f68-93efec31e49a None None] MFA Rules not processed for user `df400aee39a3498d8a26d55180075ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:06.500304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-57e175bc-4c0e-49db-9f68-93efec31e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:06.606740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 869/1723] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:07.409885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:07.721630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:07.820714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc929abb-2cd4-4088-8347-6c97c01a1245 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:07.825365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:07.826399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:07.827335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:07.827335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] PATH_INFO: `/v3/users/680e497764614a21b0182954d82bd0f9` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:07.828403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680e497764614a21b0182954d82bd0f9)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:07.831970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:08.110844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680e497764614a21b0182954d82bd0f9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:08.112792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680e497764614a21b0182954d82bd0f9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:08.121537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680e497764614a21b0182954d82bd0f9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:08.356174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680e497764614a21b0182954d82bd0f9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:08.370864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3c3455ab-3408-4b86-99f8-54302b3d5301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680e497764614a21b0182954d82bd0f9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:08.381343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 855/1724] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:07 2022] DELETE /identity/v3/users/680e497764614a21b0182954d82bd0f9 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:08.384955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:08.385420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:08.385516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:08.410971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:08.411882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:08.412170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:08.418134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:08.419692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:08.434867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] MFA Rules not processed for user `e367e391e68a477d861a515f846a8dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:08.436206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:08.518788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 870/1725] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:08.527766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 856/1726] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:08.534934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:08.539347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:08.626999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b43e5c-9302-46f5-90c0-5c6f8403d953 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:08.630951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:08.631729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:08.632082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:08.632430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:08.649959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:08.649959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:08.657178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 871/1727] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:09.425786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:09.427448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:09.451019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eb29e-800c-4755-86e2-b5b6bbf58751 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:09.466025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:09.466964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:09.467250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:09.467531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] PATH_INFO: `/v3/projects/c8323dd57f014e3b911a078f40cbda89` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:09.468889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8323dd57f014e3b911a078f40cbda89)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:09.471408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:09.631624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8323dd57f014e3b911a078f40cbda89', 'request_id': 'req-310bdfa3-4480-49e3-a42e-97d4d5fd271f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:09.641256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8323dd57f014e3b911a078f40cbda89', 'request_id': 'req-310bdfa3-4480-49e3-a42e-97d4d5fd271f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:09.712841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:09.814579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8323dd57f014e3b911a078f40cbda89', 'request_id': 'req-310bdfa3-4480-49e3-a42e-97d4d5fd271f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:09.829095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] The token cache is being invalidate because project c8323dd57f014e3b911a078f40cbda89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:09.829538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:09.831195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:09.835923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 857/1728] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:09 2022] DELETE /identity/v3/projects/c8323dd57f014e3b911a078f40cbda89 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:09.837637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:09.847576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16872516-26a0-453d-b051-319cb426c589 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:09.913909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:09.915262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:09.915262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:09.915447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] PATH_INFO: `/v3/users/2d04f40ef60d4416bd775d15d3986084` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:09.916504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d04f40ef60d4416bd775d15d3986084)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:09.918631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:10.297943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d04f40ef60d4416bd775d15d3986084'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:10.300081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d04f40ef60d4416bd775d15d3986084'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:10.301576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d04f40ef60d4416bd775d15d3986084'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:10.323780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d04f40ef60d4416bd775d15d3986084'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:10.478122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5d7bb1b2-b067-4def-b1f6-4ece5a2947e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d04f40ef60d4416bd775d15d3986084'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:10.492045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 872/1729] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:09 2022] DELETE /identity/v3/users/2d04f40ef60d4416bd775d15d3986084 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:11.107451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:11.109592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:11.131229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdfa3-4480-49e3-a42e-97d4d5fd271f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:11.201646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:11.202600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:11.202879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:11.203154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] PATH_INFO: `/v3/projects/11fa1155f7634aa5810903ca83953982` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:11.204365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fa1155f7634aa5810903ca83953982)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:11.207089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:11.574350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fa1155f7634aa5810903ca83953982', 'request_id': 'req-fd86784d-9254-4d2f-8dbc-5d849234c255'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:11.580690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fa1155f7634aa5810903ca83953982', 'request_id': 'req-fd86784d-9254-4d2f-8dbc-5d849234c255'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:11.775337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fa1155f7634aa5810903ca83953982', 'request_id': 'req-fd86784d-9254-4d2f-8dbc-5d849234c255'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:11.787043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] The token cache is being invalidate because project 11fa1155f7634aa5810903ca83953982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:11.787337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:11.787996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-fd86784d-9254-4d2f-8dbc-5d849234c255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:11.789575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 858/1730] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:11 2022] DELETE /identity/v3/projects/11fa1155f7634aa5810903ca83953982 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:11.795355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bbaa7-3fcc-4dc0-8520-7955961d0488 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:11.795802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bbaa7-3fcc-4dc0-8520-7955961d0488 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:11.795910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bbaa7-3fcc-4dc0-8520-7955961d0488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:11.814976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-885bbaa7-3fcc-4dc0-8520-7955961d0488 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:11.816632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-885bbaa7-3fcc-4dc0-8520-7955961d0488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:11.888787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 873/1731] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:11.890391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e8ad4-c4ae-4fc7-9029-75d25a06838f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:11.890685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e8ad4-c4ae-4fc7-9029-75d25a06838f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:11.890969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e8ad4-c4ae-4fc7-9029-75d25a06838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:11.909813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-665e8ad4-c4ae-4fc7-9029-75d25a06838f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:11.911139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-665e8ad4-c4ae-4fc7-9029-75d25a06838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:11.919467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 859/1732] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:11.926717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bbaa7-3fcc-4dc0-8520-7955961d0488 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:11.936596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bbaa7-3fcc-4dc0-8520-7955961d0488 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:11.941948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:11.942662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:11.942948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:11.943228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:11.944042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:11.944600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:11.946195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:11.958340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 874/1733] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:56:11.962820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e8ad4-c4ae-4fc7-9029-75d25a06838f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:11.972977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e8ad4-c4ae-4fc7-9029-75d25a06838f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:11.978510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:11.979221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:11.979502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:11.979776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:11.980698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:11.982704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:12.090044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 860/1734] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:11 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:12.094366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:12.105240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65ca9b-a227-4ba0-9497-87f06d487615 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:12.161640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:12.162356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:12.162640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:12.162911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:12.163600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:12.165525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:12.306300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 875/1735] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:12 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:12.310334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:12.320712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb426c1f-9252-495d-8ab9-0fdcae5ec2d2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:12.325309 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:12.325981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:12.326301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:12.326577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:12.327301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:12.328898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:12.342443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 861/1736] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:12.347838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:12.357852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8686599-5da9-496e-a966-dd314572b834 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:12.362599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9148fa4-3ad4-40c5-ab8e-1fc01ddf30fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:12.363356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9148fa4-3ad4-40c5-ab8e-1fc01ddf30fa admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:12.363710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9148fa4-3ad4-40c5-ab8e-1fc01ddf30fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:12.364118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9148fa4-3ad4-40c5-ab8e-1fc01ddf30fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:12.364925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9148fa4-3ad4-40c5-ab8e-1fc01ddf30fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:12.366637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9148fa4-3ad4-40c5-ab8e-1fc01ddf30fa admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:12.377266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 876/1737] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:12.386572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:12.397557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927c03a-c4df-44b7-baa3-e7b90e537936 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:12.403162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:12.404350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:12.404642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:12.404923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] PATH_INFO: `/v3/projects/1b94fffb90c24489a3f24aeb928c405b/users/c723a788dc674680a0de3a1b3c549bb9/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:12.424173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b94fffb90c24489a3f24aeb928c405b, user_id=c723a788dc674680a0de3a1b3c549bb9, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:12.425033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:12.554928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 862/1738] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:12 2022] PUT /identity/v3/projects/1b94fffb90c24489a3f24aeb928c405b/users/c723a788dc674680a0de3a1b3c549bb9/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:12.575880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93a36-5502-4d1a-87fd-eef122020b5d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:12.576178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93a36-5502-4d1a-87fd-eef122020b5d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:12.576458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b93a36-5502-4d1a-87fd-eef122020b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:12.606398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-79b93a36-5502-4d1a-87fd-eef122020b5d None None] MFA Rules not processed for user `c723a788dc674680a0de3a1b3c549bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:12.607811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-79b93a36-5502-4d1a-87fd-eef122020b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:12.686645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 877/1739] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:12.691819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:12.772832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e69b2-a9d7-407a-a800-6717e25e960b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:12.780326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:12.781023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:12.781308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:12.781588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:12.782341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:12.798320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:13.062812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 863/1740] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:12 2022] POST /identity/v3/projects => generated 385 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:13.066300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b93a36-5502-4d1a-87fd-eef122020b5d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:13.085354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b93a36-5502-4d1a-87fd-eef122020b5d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:13.164109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:13.165061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:13.165489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:13.165904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:13.166768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:13.169434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:13.421970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 878/1741] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:13 2022] POST /identity/v3/users => generated 293 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:13.428068 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:13.441014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7036b5-d5b7-44a7-bca6-78d342ff74da admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:13.447110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:13.448071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:13.448515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:13.448949 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:13.450252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:13.452515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:13.468737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 864/1742] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:13.474284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:13.485346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0792134-c1b0-43fc-b7a9-f34741d9da30 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:13.493607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc39ee-de9f-46f8-9dac-21702e883565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:13.494385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc39ee-de9f-46f8-9dac-21702e883565 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:13.494672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc39ee-de9f-46f8-9dac-21702e883565 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:13.494949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc39ee-de9f-46f8-9dac-21702e883565 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:13.495690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc39ee-de9f-46f8-9dac-21702e883565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:13.497406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc39ee-de9f-46f8-9dac-21702e883565 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:13.512418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 879/1743] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:13.518207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:13.531270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b1eb2-b36b-45f0-8e16-579620fe1b91 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:13.537928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:13.538937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:13.539392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:13.539859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] PATH_INFO: `/v3/projects/38ec98ad245f4211ab0d42aac8e4bb7e/users/7850c58ac25b4d3c9e1b7b65090dc2b0/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:13.557868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ec98ad245f4211ab0d42aac8e4bb7e, user_id=7850c58ac25b4d3c9e1b7b65090dc2b0, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:13.563690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:13.845145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 865/1744] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:13 2022] PUT /identity/v3/projects/38ec98ad245f4211ab0d42aac8e4bb7e/users/7850c58ac25b4d3c9e1b7b65090dc2b0/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:13.858273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8de76-9dfc-494c-b55a-3f30f88d1ed5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:13.858819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8de76-9dfc-494c-b55a-3f30f88d1ed5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:13.858945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8de76-9dfc-494c-b55a-3f30f88d1ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:13.919385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-02e8de76-9dfc-494c-b55a-3f30f88d1ed5 None None] MFA Rules not processed for user `7850c58ac25b4d3c9e1b7b65090dc2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:13.924818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-02e8de76-9dfc-494c-b55a-3f30f88d1ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:14.090212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 880/1745] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:14.095229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:14.232752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd9762-570b-4b4c-bd76-fbdcb608e272 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:14.237577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:14.238308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:14.238596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:14.238870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:14.239572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:14.241617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:14.445808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 866/1746] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:14 2022] POST /identity/v3/projects => generated 387 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:14.453936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8de76-9dfc-494c-b55a-3f30f88d1ed5 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:14.464030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8de76-9dfc-494c-b55a-3f30f88d1ed5 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:14.555850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:14.556585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:14.556873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:14.557396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:14.557879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:14.559924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:14.768642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 881/1747] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:14 2022] POST /identity/v3/users => generated 300 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:14.768848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:14.768848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29cadc-35e4-432a-9688-a4eb6600939a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:14.768848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3808e-5047-4c0a-abd8-65b9dec30497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:14.768848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3808e-5047-4c0a-abd8-65b9dec30497 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:14.769112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3808e-5047-4c0a-abd8-65b9dec30497 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:14.769112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3808e-5047-4c0a-abd8-65b9dec30497 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:14.769112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3808e-5047-4c0a-abd8-65b9dec30497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:14.769112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3808e-5047-4c0a-abd8-65b9dec30497 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:14.786382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 867/1748] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:14.788891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:14.804473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b1247-c5b4-490b-b7eb-59094188d451 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:14.815701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:14.820659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:14.820659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:14.820659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] PATH_INFO: `/v3/projects/da95b03785b84019b78a3c43ce52f25b/users/c546a63dbb8249cbb63e2294ecbb0579/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:14.856253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da95b03785b84019b78a3c43ce52f25b, user_id=c546a63dbb8249cbb63e2294ecbb0579, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:14.859023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:14.972868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:15.032225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41e7b4-1068-4d4b-b110-577bfb73627e None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:15.032225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41e7b4-1068-4d4b-b110-577bfb73627e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:15.032225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41e7b4-1068-4d4b-b110-577bfb73627e None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:15.032954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 868/1749] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:56:15.050833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88fe67-69a7-478f-8a25-85531535524e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:15.051129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88fe67-69a7-478f-8a25-85531535524e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:15.051704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88fe67-69a7-478f-8a25-85531535524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:15.075812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5d88fe67-69a7-478f-8a25-85531535524e None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:15.077290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5d88fe67-69a7-478f-8a25-85531535524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:15.112135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 882/1750] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:14 2022] PUT /identity/v3/projects/da95b03785b84019b78a3c43ce52f25b/users/c546a63dbb8249cbb63e2294ecbb0579/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:15.116836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:15.216615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cadc6d-9dd6-4419-b1a1-2b0d25aeec91 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:15.222671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:15.223967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:15.224557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:15.225167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:15.226428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:15.228787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:15.249157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 883/1751] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:15.249311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 869/1752] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:56:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:15.253632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:15.275529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640927c3-272b-45a6-989d-1e7f67b0be92 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:15.291510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:15.292698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:15.293247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:15.293852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] PATH_INFO: `/v3/projects/da95b03785b84019b78a3c43ce52f25b/users/c546a63dbb8249cbb63e2294ecbb0579/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:15.299807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da95b03785b84019b78a3c43ce52f25b, user_id=c546a63dbb8249cbb63e2294ecbb0579, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:15.305188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:15.340258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 884/1753] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:15 2022] PUT /identity/v3/projects/da95b03785b84019b78a3c43ce52f25b/users/c546a63dbb8249cbb63e2294ecbb0579/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:15.341925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:15.342242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:15.342529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:15.442358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] MFA Rules not processed for user `c546a63dbb8249cbb63e2294ecbb0579`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:15.444443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:15.582586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 870/1754] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:15.732570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:15.732570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:15.777159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:15.779202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:15.867821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665aa417-0979-49cf-ae3f-5a5c1cc35556 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:15.883296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:15.883984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:15.884267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:15.884543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:15.911678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:15.916559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:15.934787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 885/1755] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:15.971930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d2419-6c51-4e8d-ba39-f6f39dd763d4 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:15.988725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:15.991363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:15.993072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:15.994592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:16.018062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:16.021491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:16.039601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 871/1756] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:56:17.340864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:18.263124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:18.276558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:18.279775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e023a-98a9-4604-9e31-267ac47bd262 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:18.283993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2547247-a191-4c6a-9ffe-633918888af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:18.284871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2547247-a191-4c6a-9ffe-633918888af2 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:18.285330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2547247-a191-4c6a-9ffe-633918888af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:18.287949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2547247-a191-4c6a-9ffe-633918888af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:18.368287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2547247-a191-4c6a-9ffe-633918888af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:18.374638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2547247-a191-4c6a-9ffe-633918888af2 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:18.379740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 886/1757] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:18.876609 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:18.879487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:18.970070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde813f-38d6-4853-aef1-a9b7619007f7 service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:18.977195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:18.979399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:18.979681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:18.979971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] PATH_INFO: `/v3/users/9bb1d24279e64d9e85a3d43a1c59014e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:18.981372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb1d24279e64d9e85a3d43a1c59014e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:18.983469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:19.355375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb1d24279e64d9e85a3d43a1c59014e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:19.356096 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb1d24279e64d9e85a3d43a1c59014e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:19.363691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb1d24279e64d9e85a3d43a1c59014e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:19.506390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb1d24279e64d9e85a3d43a1c59014e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:19.525694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb1d24279e64d9e85a3d43a1c59014e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:19.533722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 872/1758] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:18 2022] DELETE /identity/v3/users/9bb1d24279e64d9e85a3d43a1c59014e => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:19.538206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:19.538736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:19.539196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:19.560307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:19.561827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:19.647991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 887/1759] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:19.670453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:19.670453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:19.730572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68097531-67ce-4cd3-be34-cdb7401fc03a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:19.734259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6180b-76db-49da-b9f8-e2bbb6d8b262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:19.735111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6180b-76db-49da-b9f8-e2bbb6d8b262 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:19.735539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6180b-76db-49da-b9f8-e2bbb6d8b262 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:19.736660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6180b-76db-49da-b9f8-e2bbb6d8b262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:19.747711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6180b-76db-49da-b9f8-e2bbb6d8b262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:19.749180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6180b-76db-49da-b9f8-e2bbb6d8b262 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:19.760435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 873/1760] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:20.550279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:20.553548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:20.574326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d80f8-3023-46dc-b79a-e5eabee219a5 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:20.583023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:20.585123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:20.585604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:20.585902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] PATH_INFO: `/v3/projects/d8aec72d87bb41c68e1337dcad262434` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:20.587690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8aec72d87bb41c68e1337dcad262434)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:20.589705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:20.995156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8aec72d87bb41c68e1337dcad262434', 'request_id': 'req-d67138fb-f141-4e1a-92a1-945da8292a84'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:21.174238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8aec72d87bb41c68e1337dcad262434', 'request_id': 'req-d67138fb-f141-4e1a-92a1-945da8292a84'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:21.180837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8aec72d87bb41c68e1337dcad262434', 'request_id': 'req-d67138fb-f141-4e1a-92a1-945da8292a84'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:21.200794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] The token cache is being invalidate because project d8aec72d87bb41c68e1337dcad262434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:21.201168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:21.201978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d67138fb-f141-4e1a-92a1-945da8292a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:21.205910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 888/1761] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:20 2022] DELETE /identity/v3/projects/d8aec72d87bb41c68e1337dcad262434 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:21.211089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21d7d-dab7-469b-8b12-8c91922b2087 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:21.211591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21d7d-dab7-469b-8b12-8c91922b2087 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:21.211699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21d7d-dab7-469b-8b12-8c91922b2087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:21.233206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a0a21d7d-dab7-469b-8b12-8c91922b2087 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:21.234737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a0a21d7d-dab7-469b-8b12-8c91922b2087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:21.334140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 874/1762] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:21.335406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e6e9-0817-47e6-981e-5a59435dd97d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:21.342505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e6e9-0817-47e6-981e-5a59435dd97d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:21.342505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e6e9-0817-47e6-981e-5a59435dd97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:21.376879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-fec7e6e9-0817-47e6-981e-5a59435dd97d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:21.382897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fec7e6e9-0817-47e6-981e-5a59435dd97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:21.408691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 889/1763] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:21.409923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21d7d-dab7-469b-8b12-8c91922b2087 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:21.425680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21d7d-dab7-469b-8b12-8c91922b2087 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:21.435695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b66d9-3b08-487c-a3a7-a76c3dd14c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:21.436653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b66d9-3b08-487c-a3a7-a76c3dd14c8d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:21.437153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b66d9-3b08-487c-a3a7-a76c3dd14c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:21.439068 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b66d9-3b08-487c-a3a7-a76c3dd14c8d admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:21.441102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b66d9-3b08-487c-a3a7-a76c3dd14c8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:21.441813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b66d9-3b08-487c-a3a7-a76c3dd14c8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:21.443852 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b66d9-3b08-487c-a3a7-a76c3dd14c8d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:21.459210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 875/1764] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:56:21.463348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e6e9-0817-47e6-981e-5a59435dd97d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:21.480692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e6e9-0817-47e6-981e-5a59435dd97d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:21.489166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:21.489858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:21.490255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:21.490547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:21.491297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:21.493266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:21.610982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40bf36-79e3-483a-9af4-6a29a8e419d5 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:21.611373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40bf36-79e3-483a-9af4-6a29a8e419d5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:21.611751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40bf36-79e3-483a-9af4-6a29a8e419d5 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:21.613299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 876/1765] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:56:21.642772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:21.643053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:21.643339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:21.680757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:21.682769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:21.746201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 890/1766] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:21 2022] POST /identity/v3/projects => generated 367 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:21.750343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:21.826621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 877/1767] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:56:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:21.853810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5028a-a4b0-46b9-968e-66b18ac58904 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:21.859105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:21.860600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:21.860888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:21.863688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:21.863688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:21.863933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:22.052068 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:22.064301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:22.067926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 891/1768] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:21 2022] POST /identity/v3/users => generated 284 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:22.074517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:22.119479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab37ed-0268-4e56-a89a-76a557d30350 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:22.120001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d590c-f968-4f5e-bae8-a31f71b32820 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:22.131133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:22.131844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:22.132132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:22.132409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:22.133153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:22.140395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:22.166063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 892/1769] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:22.167014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:22.189368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b522d8-08b7-4dc0-948e-1bbfe60dbcec admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:22.201548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:22.203675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:22.206013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:22.207550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:22.207550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:22.209509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:22.217635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621f9e8-e9f3-4c55-9cc7-290641d7add4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:22.218829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621f9e8-e9f3-4c55-9cc7-290641d7add4 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:22.219760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621f9e8-e9f3-4c55-9cc7-290641d7add4 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:22.221108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621f9e8-e9f3-4c55-9cc7-290641d7add4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:22.231657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621f9e8-e9f3-4c55-9cc7-290641d7add4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:22.237756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 893/1770] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:22.240320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621f9e8-e9f3-4c55-9cc7-290641d7add4 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:22.241825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:22.263024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f04b73-c4b3-40ca-93fe-823ee101c341 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:22.269616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:22.271234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:22.271685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:22.272122 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] PATH_INFO: `/v3/projects/792f432ccdb146dd816db9d0f467111f/users/504f5d1e4d6d477eb2c1978c1dce2699/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:22.282372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 878/1771] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:56:22.292653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792f432ccdb146dd816db9d0f467111f, user_id=504f5d1e4d6d477eb2c1978c1dce2699, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:22.295782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:22.458552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 894/1772] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:22 2022] PUT /identity/v3/projects/792f432ccdb146dd816db9d0f467111f/users/504f5d1e4d6d477eb2c1978c1dce2699/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:22.469809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503c15-8d90-4c40-aa5e-e2b4d3baa7a5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:22.469809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503c15-8d90-4c40-aa5e-e2b4d3baa7a5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:22.470587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503c15-8d90-4c40-aa5e-e2b4d3baa7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:22.510073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-56503c15-8d90-4c40-aa5e-e2b4d3baa7a5 None None] MFA Rules not processed for user `504f5d1e4d6d477eb2c1978c1dce2699`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:22.511324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-56503c15-8d90-4c40-aa5e-e2b4d3baa7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:22.665056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 879/1773] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:22.671770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:22.747545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c1d5d-d206-4a3d-a1ab-df6e8173d083 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:22.753247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:22.757254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:22.757543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:22.757824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:22.758570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:22.760658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:22.963155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 895/1774] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:22 2022] POST /identity/v3/projects => generated 377 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:22.967274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56503c15-8d90-4c40-aa5e-e2b4d3baa7a5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:22.978765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56503c15-8d90-4c40-aa5e-e2b4d3baa7a5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.037576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.038278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.038568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.038843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.039523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:23.041429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:23.150520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:23.191310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 880/1775] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:22 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:23.195250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:23.206244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c53fcd-d16d-4974-9f2e-5539099fa6cf admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.210899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.211573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.211873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.212141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.212969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:23.219966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:23.230296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 896/1776] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:23.233205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:23.246394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeca85-5196-4b6c-9009-50a6e948ecd5 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.251804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15450b7-a81e-4bad-b948-aa92aa63c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.255915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15450b7-a81e-4bad-b948-aa92aa63c934 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.255915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15450b7-a81e-4bad-b948-aa92aa63c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.255915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15450b7-a81e-4bad-b948-aa92aa63c934 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.255915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15450b7-a81e-4bad-b948-aa92aa63c934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:23.255915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15450b7-a81e-4bad-b948-aa92aa63c934 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:23.269721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 881/1777] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:23.270931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:23.281016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6744f-238c-4a81-8fb5-52e354018c00 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.285843 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.286762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.287128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.287301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] PATH_INFO: `/v3/projects/3be1cba5128644599445e43a765ba9df/users/680e7a19ad284632ae351548ad1e5f79/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.303099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be1cba5128644599445e43a765ba9df, user_id=680e7a19ad284632ae351548ad1e5f79, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:23.305796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:23.407963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 897/1778] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:23 2022] PUT /identity/v3/projects/3be1cba5128644599445e43a765ba9df/users/680e7a19ad284632ae351548ad1e5f79/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:23.419323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ec536-19f2-4eb7-bcc5-a281a0fdaf12 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.419614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ec536-19f2-4eb7-bcc5-a281a0fdaf12 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.419891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ec536-19f2-4eb7-bcc5-a281a0fdaf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.445408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-f78ec536-19f2-4eb7-bcc5-a281a0fdaf12 None None] MFA Rules not processed for user `680e7a19ad284632ae351548ad1e5f79`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:23.446624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f78ec536-19f2-4eb7-bcc5-a281a0fdaf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:23.508653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 882/1779] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:23.513206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:23.579451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d5a64-e1ff-4522-8000-3fbceccd9218 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.584207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.584874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.585158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.585439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.586265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:23.588192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:23.725478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 898/1780] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:23 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:23.729617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ec536-19f2-4eb7-bcc5-a281a0fdaf12 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:23.741835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ec536-19f2-4eb7-bcc5-a281a0fdaf12 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.835625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.836407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.836765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.837112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.837882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:23.839865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:23.932580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 883/1781] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:23 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:23.936402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:23.947270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf061f1-4625-42f2-9dab-2e62235fa55a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.951763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.952430 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.953395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.953798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:23.954754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:23.956365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:23.966797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 899/1782] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:23.971045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:23.981409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397d11c-4b72-4b5b-85b6-b5d24f331185 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:23.986107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:23.987026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:23.987303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:23.987576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] PATH_INFO: `/v3/projects/defb778477534329b41b235af1769038/users/5890ec9dd94c4927816132c6179c168c/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:24.002765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=defb778477534329b41b235af1769038, user_id=5890ec9dd94c4927816132c6179c168c, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:24.013057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:24.115269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 884/1783] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:23 2022] PUT /identity/v3/projects/defb778477534329b41b235af1769038/users/5890ec9dd94c4927816132c6179c168c/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:24.119188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:24.129029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8814b5b-261b-4d23-8fd7-9925abeb1bb5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:24.184609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda01336-9262-465f-bbc4-322e34216ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:24.185277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda01336-9262-465f-bbc4-322e34216ddf admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:24.185560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda01336-9262-465f-bbc4-322e34216ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:24.185834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda01336-9262-465f-bbc4-322e34216ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:24.186582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda01336-9262-465f-bbc4-322e34216ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:24.188748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda01336-9262-465f-bbc4-322e34216ddf admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:24.198450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 900/1784] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:24.203339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:24.213526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a612-5238-4b2b-94aa-600354d37014 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:24.218050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:24.219799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:24.220085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:24.220388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] PATH_INFO: `/v3/projects/defb778477534329b41b235af1769038/users/5890ec9dd94c4927816132c6179c168c/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:24.222525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] RBAC: Authorizing `identity:create_grant(project_id=defb778477534329b41b235af1769038, user_id=5890ec9dd94c4927816132c6179c168c, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:24.225019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:24.235733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 885/1785] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:24 2022] PUT /identity/v3/projects/defb778477534329b41b235af1769038/users/5890ec9dd94c4927816132c6179c168c/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:24.245713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:24.246013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:24.246327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:24.273651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] MFA Rules not processed for user `5890ec9dd94c4927816132c6179c168c`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:24.275218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:24.332863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:24.351746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 901/1786] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:24.363604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:24.365316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:24.425710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d19a9-a224-495e-9c38-ee6db27a339d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:24.429776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:24.430455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:24.430739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:24.431009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:24.503517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:24.504990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:24.514719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 886/1787] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:26.251536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:26.280590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:27.188166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:27.210740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:27.520431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:28.547404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:28.646794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:28.733393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:28.734856 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:28.748180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81342a7-356d-4e59-a1ca-5be06479cc35 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:28.751731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:28.752399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:28.752674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:28.753572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:28.814871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:28.816459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:28.826094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 902/1788] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:30.759338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:30.769552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36638-33b4-4c8a-a5bc-113ce4ab4884 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:30.774485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:30.782326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:30.782326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:30.782326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:30.786692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:30.788143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:30.800212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 887/1789] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:31.989852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:32.106848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:32.939657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:32.950357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e37d59-3fb8-4c60-9bc6-09becfd15fc0 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:32.952974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7961c-593d-4532-ab2c-0829c097ab40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:32.953624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7961c-593d-4532-ab2c-0829c097ab40 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:32.953908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7961c-593d-4532-ab2c-0829c097ab40 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:32.954213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7961c-593d-4532-ab2c-0829c097ab40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:32.955995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f7b7961c-593d-4532-ab2c-0829c097ab40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:33.016563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7961c-593d-4532-ab2c-0829c097ab40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:33.017957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7961c-593d-4532-ab2c-0829c097ab40 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:33.029063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 903/1790] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:34.379502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:35.637214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:36.375023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:36.497335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7963-5c97-43b8-b717-aefae8c9005a service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:36.512813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:36.512813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:36.512813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:36.512813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] PATH_INFO: `/v3/users/504f5d1e4d6d477eb2c1978c1dce2699` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:36.513304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=504f5d1e4d6d477eb2c1978c1dce2699)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:36.513862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:36.853331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:36.867025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504f5d1e4d6d477eb2c1978c1dce2699'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:36.867745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504f5d1e4d6d477eb2c1978c1dce2699'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:36.875528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504f5d1e4d6d477eb2c1978c1dce2699'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:36.996086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504f5d1e4d6d477eb2c1978c1dce2699'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:37.010797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504f5d1e4d6d477eb2c1978c1dce2699'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:37.018696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 888/1791] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:36 2022] DELETE /identity/v3/users/504f5d1e4d6d477eb2c1978c1dce2699 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:37.023859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:37.024344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:37.024723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:37.045366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:37.050105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:37.109592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 904/1792] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:37.119701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:37.121152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:37.183916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651679aa-264d-4927-aa03-797b9aa71993 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:37.188187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:37.188844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:37.189132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:37.189414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:37.199391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:37.200866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:37.210717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 889/1793] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:37.667794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:37.669202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:37.682830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa6eb0-d4ca-40d6-bfbb-bdfa81a63466 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:37.688039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:37.689106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:37.689541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:37.689963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] PATH_INFO: `/v3/projects/792f432ccdb146dd816db9d0f467111f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:37.691433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=792f432ccdb146dd816db9d0f467111f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:37.693527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:37.903932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792f432ccdb146dd816db9d0f467111f', 'request_id': 'req-50fe7787-e541-4c42-b389-a550e6512261'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.034632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792f432ccdb146dd816db9d0f467111f', 'request_id': 'req-50fe7787-e541-4c42-b389-a550e6512261'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.041607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792f432ccdb146dd816db9d0f467111f', 'request_id': 'req-50fe7787-e541-4c42-b389-a550e6512261'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.053477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] The token cache is being invalidate because project 792f432ccdb146dd816db9d0f467111f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:38.053775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.054471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.056353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 905/1794] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:37 2022] DELETE /identity/v3/projects/792f432ccdb146dd816db9d0f467111f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:38.060526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:38.070240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97520-49a7-42b8-aad4-432cb3b1affc service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:38.141030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:38.143659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:38.143944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:38.144217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] PATH_INFO: `/v3/users/680e7a19ad284632ae351548ad1e5f79` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:38.145555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680e7a19ad284632ae351548ad1e5f79)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:38.147784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:38.242675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:38.245871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:38.270986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe7787-e541-4c42-b389-a550e6512261 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:38.277540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:38.278542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:38.278908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:38.279251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] PATH_INFO: `/v3/users/c723a788dc674680a0de3a1b3c549bb9` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:38.280528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c723a788dc674680a0de3a1b3c549bb9)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:38.282676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:38.400259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680e7a19ad284632ae351548ad1e5f79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.401390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680e7a19ad284632ae351548ad1e5f79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.417838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680e7a19ad284632ae351548ad1e5f79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.557570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680e7a19ad284632ae351548ad1e5f79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.567169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:38.588078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c91e35cd-a42f-468a-bff9-4817ef5f33e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680e7a19ad284632ae351548ad1e5f79'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.588270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c723a788dc674680a0de3a1b3c549bb9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.590518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c723a788dc674680a0de3a1b3c549bb9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.593093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c723a788dc674680a0de3a1b3c549bb9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.606923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 890/1795] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:38 2022] DELETE /identity/v3/users/680e7a19ad284632ae351548ad1e5f79 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:38.613040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c723a788dc674680a0de3a1b3c549bb9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.697821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c723a788dc674680a0de3a1b3c549bb9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:38.730912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 906/1796] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:38 2022] DELETE /identity/v3/users/c723a788dc674680a0de3a1b3c549bb9 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:38.735479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:38.735775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:38.736055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:38.768108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:38.769331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:38.843331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 891/1797] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:38.881290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:38.882873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:38.951270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab05b-5d2d-4d8a-bc1e-35bda1f5e542 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:38.956062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:38.957282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:38.957742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:38.958164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:38.979511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:38.981234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:38.994506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 907/1798] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:39.569894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:39.572245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:39.594379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4681800-ecf2-4406-bc97-35a0e327ea2d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:39.596195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:39.597715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:39.598236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:39.598927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] PATH_INFO: `/v3/projects/1b94fffb90c24489a3f24aeb928c405b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:39.600425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b94fffb90c24489a3f24aeb928c405b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:39.603436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:39.622672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:39.624501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:39.647699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a18b-7b7b-46df-b0f7-06bd25d0ae57 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:39.666165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:39.674743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:39.674743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:39.674743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] PATH_INFO: `/v3/projects/3be1cba5128644599445e43a765ba9df` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:39.674743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be1cba5128644599445e43a765ba9df)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:39.675963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:39.823976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b94fffb90c24489a3f24aeb928c405b', 'request_id': 'req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:39.834304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b94fffb90c24489a3f24aeb928c405b', 'request_id': 'req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:39.921391 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be1cba5128644599445e43a765ba9df', 'request_id': 'req-85ebfbe0-3d46-4e46-9fec-f275ab508efb'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:39.994707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b94fffb90c24489a3f24aeb928c405b', 'request_id': 'req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.014501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] The token cache is being invalidate because project 1b94fffb90c24489a3f24aeb928c405b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:40.014797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.018600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.021316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 892/1799] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:39 2022] DELETE /identity/v3/projects/1b94fffb90c24489a3f24aeb928c405b => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:40.025503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:40.028221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:40.074043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be1cba5128644599445e43a765ba9df', 'request_id': 'req-85ebfbe0-3d46-4e46-9fec-f275ab508efb'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.080727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be1cba5128644599445e43a765ba9df', 'request_id': 'req-85ebfbe0-3d46-4e46-9fec-f275ab508efb'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.104596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] The token cache is being invalidate because project 3be1cba5128644599445e43a765ba9df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:40.105229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.106029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.111002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 908/1800] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:39 2022] DELETE /identity/v3/projects/3be1cba5128644599445e43a765ba9df => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:40.114233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:40.115958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:40.116626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22a13e-c91d-41fc-90c2-7a24ef316cb6 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:40.198601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:40.199853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebfbe0-3d46-4e46-9fec-f275ab508efb admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:40.200415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:40.200705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:40.200980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] PATH_INFO: `/v3/users/7850c58ac25b4d3c9e1b7b65090dc2b0` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:40.203039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7850c58ac25b4d3c9e1b7b65090dc2b0)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:40.205348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:40.209812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:40.211836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:40.212325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:40.213195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] PATH_INFO: `/v3/users/5890ec9dd94c4927816132c6179c168c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:40.217817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5890ec9dd94c4927816132c6179c168c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:40.217817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:40.502002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7850c58ac25b4d3c9e1b7b65090dc2b0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.503683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7850c58ac25b4d3c9e1b7b65090dc2b0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.520966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7850c58ac25b4d3c9e1b7b65090dc2b0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.540284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5890ec9dd94c4927816132c6179c168c'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.544050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5890ec9dd94c4927816132c6179c168c'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.544050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5890ec9dd94c4927816132c6179c168c'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.564605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5890ec9dd94c4927816132c6179c168c'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.736680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7850c58ac25b4d3c9e1b7b65090dc2b0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.759867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7850c58ac25b4d3c9e1b7b65090dc2b0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.768653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 893/1801] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:40 2022] DELETE /identity/v3/users/7850c58ac25b4d3c9e1b7b65090dc2b0 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:40.890835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5890ec9dd94c4927816132c6179c168c'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:40.905127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 909/1802] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:40 2022] DELETE /identity/v3/users/5890ec9dd94c4927816132c6179c168c => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:40.931431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:40.999829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:41.744490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:41.746316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:41.780360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:41.782634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:41.896180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605c097-c0d6-4ed1-a067-bf21d793056d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:41.925315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bac61-543a-457b-942d-8c0b74589e68 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:41.931476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:41.932398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:41.932688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:41.933001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] PATH_INFO: `/v3/projects/38ec98ad245f4211ab0d42aac8e4bb7e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:41.934412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ec98ad245f4211ab0d42aac8e4bb7e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:41.937067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:41.976675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:41.977555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:41.977839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:41.978115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] PATH_INFO: `/v3/projects/defb778477534329b41b235af1769038` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:41.979342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] RBAC: Authorizing `identity:delete_project(project_id=defb778477534329b41b235af1769038)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:41.981322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.058671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ec98ad245f4211ab0d42aac8e4bb7e', 'request_id': 'req-34cef9cc-48c6-4bca-9108-20c8028fc251'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.091243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'defb778477534329b41b235af1769038', 'request_id': 'req-7d5dbdbb-65bc-4106-a11e-e00c4523499d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.097897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'defb778477534329b41b235af1769038', 'request_id': 'req-7d5dbdbb-65bc-4106-a11e-e00c4523499d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.146986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ec98ad245f4211ab0d42aac8e4bb7e', 'request_id': 'req-34cef9cc-48c6-4bca-9108-20c8028fc251'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.154445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ec98ad245f4211ab0d42aac8e4bb7e', 'request_id': 'req-34cef9cc-48c6-4bca-9108-20c8028fc251'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.167434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] The token cache is being invalidate because project 38ec98ad245f4211ab0d42aac8e4bb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:42.167911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.168740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.170324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 910/1803] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:41 2022] DELETE /identity/v3/projects/38ec98ad245f4211ab0d42aac8e4bb7e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:42.174313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.175870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:42.245255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'defb778477534329b41b235af1769038', 'request_id': 'req-7d5dbdbb-65bc-4106-a11e-e00c4523499d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.257824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef9cc-48c6-4bca-9108-20c8028fc251 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.260602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] The token cache is being invalidate because project defb778477534329b41b235af1769038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:42.260954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.261938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7d5dbdbb-65bc-4106-a11e-e00c4523499d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.263677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 894/1804] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:41 2022] DELETE /identity/v3/projects/defb778477534329b41b235af1769038 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:42.269565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286dc87-c213-4b73-931a-5ca6a0cb73ff None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.270052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286dc87-c213-4b73-931a-5ca6a0cb73ff None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.270527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286dc87-c213-4b73-931a-5ca6a0cb73ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.302769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7286dc87-c213-4b73-931a-5ca6a0cb73ff None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:42.304773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7286dc87-c213-4b73-931a-5ca6a0cb73ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:42.366292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.367232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.367522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.367798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] PATH_INFO: `/v3/users/c546a63dbb8249cbb63e2294ecbb0579` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.369238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c546a63dbb8249cbb63e2294ecbb0579)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.371259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.383320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 895/1805] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:42.391782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0950955-9aee-482d-8658-4bfd04b92750 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.392257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0950955-9aee-482d-8658-4bfd04b92750 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.392679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0950955-9aee-482d-8658-4bfd04b92750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.415356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e0950955-9aee-482d-8658-4bfd04b92750 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:42.416702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e0950955-9aee-482d-8658-4bfd04b92750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:42.424919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 896/1806] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:42.430275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0950955-9aee-482d-8658-4bfd04b92750 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.438951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:42.439675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0950955-9aee-482d-8658-4bfd04b92750 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.445007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.445703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.446091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.446458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.447267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:42.447644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.449339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.461440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 897/1807] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:56:42.465596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.475200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1803cf-4a05-404e-8afc-c87e4af2a88e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.479991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.480645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.480927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.481209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.481896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.483846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.571878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 898/1808] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:42.575797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.651826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8413f-8bd2-4c5a-9718-0933fbe741f4 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.656555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.657234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.657513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.657787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.658508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.660458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.692831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c546a63dbb8249cbb63e2294ecbb0579'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.693585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c546a63dbb8249cbb63e2294ecbb0579'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.698366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c546a63dbb8249cbb63e2294ecbb0579'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.714815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c546a63dbb8249cbb63e2294ecbb0579'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.779839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 899/1809] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:42 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:42.781134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.782967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:42.797085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c546a63dbb8249cbb63e2294ecbb0579'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:42.811511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fb85-e2c1-46cb-a6ef-2d3302ed57d9 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.820728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.822319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.823001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.823886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.825459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.827068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 911/1810] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:42 2022] DELETE /identity/v3/users/c546a63dbb8249cbb63e2294ecbb0579 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:42.829149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.842940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 900/1811] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:42.852363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.932651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e7c93-fb25-43c9-a0ad-b02852c3cb91 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.938869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e0289-2ac5-438d-84eb-53404a30d158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.939690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e0289-2ac5-438d-84eb-53404a30d158 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.940149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e0289-2ac5-438d-84eb-53404a30d158 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.940471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e0289-2ac5-438d-84eb-53404a30d158 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:42.941416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e0289-2ac5-438d-84eb-53404a30d158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:42.943186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e0289-2ac5-438d-84eb-53404a30d158 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:42.965421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 912/1812] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:42.969436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:42.981032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba01eb-7cec-4d0e-8b9a-363d8caa635f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:42.986051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:42.987009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:42.987297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:42.987574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] PATH_INFO: `/v3/projects/c5f62642d18d40329dde836786760296/users/18630c7f125040718fbcc651348941f6/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.004815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f62642d18d40329dde836786760296, user_id=18630c7f125040718fbcc651348941f6, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.008124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.114948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 901/1813] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:42 2022] PUT /identity/v3/projects/c5f62642d18d40329dde836786760296/users/18630c7f125040718fbcc651348941f6/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:43.125716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.125968 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.126217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.156266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] MFA Rules not processed for user `18630c7f125040718fbcc651348941f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:43.157482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.222463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 913/1814] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:43.227206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.288602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02b644-f4ee-471c-b3c8-11305054cfc2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.293615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.293950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.294261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.294605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.295257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.297173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.359685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.361147 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.378148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe0c30-fb7d-4447-84bf-913c54afd77b None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.382980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.383961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.384135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.384413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] PATH_INFO: `/v3/projects/da95b03785b84019b78a3c43ce52f25b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.386024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da95b03785b84019b78a3c43ce52f25b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.388192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.439268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 902/1815] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:43.443237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.457530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:43.497994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da95b03785b84019b78a3c43ce52f25b', 'request_id': 'req-e1c94152-a824-4ab4-8278-c4501a08dd2d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:43.509007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ede7f-7316-488f-af6c-7063cd50a0c0 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.567538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.568220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.568507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.569344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.570046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.572048 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.586686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da95b03785b84019b78a3c43ce52f25b', 'request_id': 'req-e1c94152-a824-4ab4-8278-c4501a08dd2d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:43.592841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da95b03785b84019b78a3c43ce52f25b', 'request_id': 'req-e1c94152-a824-4ab4-8278-c4501a08dd2d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:43.604203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] The token cache is being invalidate because project da95b03785b84019b78a3c43ce52f25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:43.604486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:43.605135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e1c94152-a824-4ab4-8278-c4501a08dd2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:43.606741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 914/1816] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:43 2022] DELETE /identity/v3/projects/da95b03785b84019b78a3c43ce52f25b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:43.612745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95696304-2f91-40d7-8c6d-6692dd34fdbd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.613031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95696304-2f91-40d7-8c6d-6692dd34fdbd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.613308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95696304-2f91-40d7-8c6d-6692dd34fdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.632872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-95696304-2f91-40d7-8c6d-6692dd34fdbd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:43.634020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-95696304-2f91-40d7-8c6d-6692dd34fdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.642613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 915/1817] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:43.652278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403203f3-d953-4279-8ace-4b79b68f2f90 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.652566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403203f3-d953-4279-8ace-4b79b68f2f90 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.659618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403203f3-d953-4279-8ace-4b79b68f2f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.671276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-403203f3-d953-4279-8ace-4b79b68f2f90 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:43.672478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-403203f3-d953-4279-8ace-4b79b68f2f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.681043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 916/1818] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:43.685274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403203f3-d953-4279-8ace-4b79b68f2f90 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.689358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 903/1819] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:43.692988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.694882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403203f3-d953-4279-8ace-4b79b68f2f90 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.695324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.699627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.700283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.700563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.700840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.701589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:43.701895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.703682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.707125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcc9e1-e37e-4c26-9040-5924e71489b2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.712294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.712961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.721129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.721129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.721129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.721129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.721455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 917/1820] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:56:43.723311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.726829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 904/1821] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:43.730555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.733922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d426cc4-4f7e-4457-8f3f-8c722150242c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.738929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.739578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.739835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9877fd2-e5e6-4db5-8249-bc4af006abad admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.739933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.740115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.741564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.743769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.744977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.745624 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.745908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.746259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.746961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.748534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.759074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 905/1822] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:43.768892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.779056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab601f3c-ff2c-4cca-845c-c9555caf571b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:43.784227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5d970-f5a3-4d5d-9e33-0f16ef2794e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:43.785213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5d970-f5a3-4d5d-9e33-0f16ef2794e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.785494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5d970-f5a3-4d5d-9e33-0f16ef2794e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.785843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5d970-f5a3-4d5d-9e33-0f16ef2794e1 admin admin] PATH_INFO: `/v3/projects/fac25ade60bb48b5832b21064903566c/users/a2c111337a1e4b92ae8b7cdd06c04946/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.802129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5d970-f5a3-4d5d-9e33-0f16ef2794e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac25ade60bb48b5832b21064903566c, user_id=a2c111337a1e4b92ae8b7cdd06c04946, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:43.806099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5d970-f5a3-4d5d-9e33-0f16ef2794e1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:43.881074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 918/1823] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:43.884833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:43.893676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 906/1824] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:43 2022] PUT /identity/v3/projects/fac25ade60bb48b5832b21064903566c/users/a2c111337a1e4b92ae8b7cdd06c04946/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:43.903009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fe1ce-5094-4d0e-ad35-1c02d198a2bb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:43.903300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fe1ce-5094-4d0e-ad35-1c02d198a2bb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:43.903576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fe1ce-5094-4d0e-ad35-1c02d198a2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:43.932696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-952fe1ce-5094-4d0e-ad35-1c02d198a2bb None None] MFA Rules not processed for user `a2c111337a1e4b92ae8b7cdd06c04946`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:43.934060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-952fe1ce-5094-4d0e-ad35-1c02d198a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:43.959948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640788a8-658b-4f6c-ba95-d43cf6c0d9b0 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.005572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 907/1825] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:44.010567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fe1ce-5094-4d0e-ad35-1c02d198a2bb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.022599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.023291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.023578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.023859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.024558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.026564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.073975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fe1ce-5094-4d0e-ad35-1c02d198a2bb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.078691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.079345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.079628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.079904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.080587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.082583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.175458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 919/1826] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:43 2022] POST /identity/v3/users => generated 292 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:44.179929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.189272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a5ab-7b2e-4b1e-9c1c-8651b3016115 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.196228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.196892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.197175 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.197453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.198197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.199904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.210527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 920/1827] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:44.214427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.224200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f4e55-1f44-4d3e-93ad-e3327db1b7c5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.228845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.229467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.229755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.230052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.230798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.232361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.245964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 921/1828] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:44.247113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 908/1829] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:44 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:44.290688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.290861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.300341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68302819-3dfa-477c-a321-a88f992d71b2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.352777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bcdef-2e6d-4e35-abef-4c455ec4ef88 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.357706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.358418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.358705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.358948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634fa6-aa03-4dcc-afdc-9e6f1e0edf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.359116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.359837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634fa6-aa03-4dcc-afdc-9e6f1e0edf96 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.359925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.360123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634fa6-aa03-4dcc-afdc-9e6f1e0edf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.360405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634fa6-aa03-4dcc-afdc-9e6f1e0edf96 admin admin] PATH_INFO: `/v3/projects/a81d0a7fbd5742059a7de18a6b09d316/users/a0916560b4f6454495eae2c804e9923b/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.362027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.375576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634fa6-aa03-4dcc-afdc-9e6f1e0edf96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a81d0a7fbd5742059a7de18a6b09d316, user_id=a0916560b4f6454495eae2c804e9923b, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.378324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634fa6-aa03-4dcc-afdc-9e6f1e0edf96 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.752177 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 909/1830] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:44 2022] POST /identity/v3/users => generated 298 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:44.808455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:44.808455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 922/1831] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:44 2022] PUT /identity/v3/projects/a81d0a7fbd5742059a7de18a6b09d316/users/a0916560b4f6454495eae2c804e9923b/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:44.811401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.820769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206b1c9-c45a-458c-896d-a5f8e90936ef None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.821060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206b1c9-c45a-458c-896d-a5f8e90936ef None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.821334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206b1c9-c45a-458c-896d-a5f8e90936ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.822015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b08a87-388b-4137-bdc4-db0c461d585c admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.848805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b206b1c9-c45a-458c-896d-a5f8e90936ef None None] MFA Rules not processed for user `a0916560b4f6454495eae2c804e9923b`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:44.851577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b206b1c9-c45a-458c-896d-a5f8e90936ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:44.887196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.888214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.888723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.889444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.898681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.900395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.929706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 910/1832] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:44.933407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.945786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458e2a5-5aa8-4fe1-ad51-1f98f8836fde admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.954252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 923/1833] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:44.955233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206b1c9-c45a-458c-896d-a5f8e90936ef None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:44.955676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.957381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.957831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.959488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] PATH_INFO: `/v3/projects/d780bf4ae9134800b33c940b76c7edd7/users/2b90a098170a4e0a89de44e513301871/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.964669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:44.970077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206b1c9-c45a-458c-896d-a5f8e90936ef None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:44.977954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:44.978491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:44.978777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:44.979052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:44.979752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.981707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:44.986786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d780bf4ae9134800b33c940b76c7edd7, user_id=2b90a098170a4e0a89de44e513301871, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:44.989863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.126754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 911/1834] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:44 2022] PUT /identity/v3/projects/d780bf4ae9134800b33c940b76c7edd7/users/2b90a098170a4e0a89de44e513301871/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:45.131631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 924/1835] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:44 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:45.131790 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.143288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.202054 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d020d-6c31-4246-ad4e-bde2941a6c0a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.206939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.207683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.208039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.208388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.209343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad72887-c654-4633-820a-43524e4a4688 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.209913 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.211565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.214757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.215263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.215546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.215822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.216517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.218496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.223252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 912/1836] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:45.226198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.235758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00035d-a9da-4254-9307-4f6f02ee28b1 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.240456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4def3-ca3c-469f-8f5c-8622a009bf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.241353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4def3-ca3c-469f-8f5c-8622a009bf0b admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.241636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4def3-ca3c-469f-8f5c-8622a009bf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.241910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4def3-ca3c-469f-8f5c-8622a009bf0b admin admin] PATH_INFO: `/v3/projects/d780bf4ae9134800b33c940b76c7edd7/users/2b90a098170a4e0a89de44e513301871/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.243939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4def3-ca3c-469f-8f5c-8622a009bf0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d780bf4ae9134800b33c940b76c7edd7, user_id=2b90a098170a4e0a89de44e513301871, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.246698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4def3-ca3c-469f-8f5c-8622a009bf0b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.257681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 913/1837] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:45 2022] PUT /identity/v3/projects/d780bf4ae9134800b33c940b76c7edd7/users/2b90a098170a4e0a89de44e513301871/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:45.268084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.268374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.268657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.295012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] MFA Rules not processed for user `2b90a098170a4e0a89de44e513301871`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:45.296214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:45.335208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 925/1838] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:45 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:45.346546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.348484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96641aa-3748-4966-bb3f-46345a0911f1 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.364124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 914/1839] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:45.375541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.376921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:45.406068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.406882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.407169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.407441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.408168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.409808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.422787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 926/1840] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:45.424319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.434512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235caf5-c557-4171-ac70-a432608c4b25 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.435132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226b4f-215e-4516-b888-b5908a2ba325 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.438769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.439324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.439456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.439660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.439941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.440198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.440477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.440747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] PATH_INFO: `/v3/projects/89ca7e103b664fe8bba10415dc767bf7/users/50dbee1565814083b1af91b3d1582227/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.443187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:45.457276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ca7e103b664fe8bba10415dc767bf7, user_id=50dbee1565814083b1af91b3d1582227, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.460687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.502076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.503699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.513262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 915/1841] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:56:45.532792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 927/1842] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:45 2022] PUT /identity/v3/projects/89ca7e103b664fe8bba10415dc767bf7/users/50dbee1565814083b1af91b3d1582227/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:45.533730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.535062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:45.546726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f51448-efa0-4594-bbaf-8d468e47f58b service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.610776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bbf0c-f8bd-4506-9190-00fdf9894ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.611452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bbf0c-f8bd-4506-9190-00fdf9894ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.611739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bbf0c-f8bd-4506-9190-00fdf9894ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.612059 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bbf0c-f8bd-4506-9190-00fdf9894ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.612790 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bbf0c-f8bd-4506-9190-00fdf9894ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.614436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bbf0c-f8bd-4506-9190-00fdf9894ad5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.626223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 916/1843] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:45.629562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.640314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a7ac-9e4b-4165-9ef1-09b73092822b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.645071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.646580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.647110 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.647336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] PATH_INFO: `/v3/projects/89ca7e103b664fe8bba10415dc767bf7/users/50dbee1565814083b1af91b3d1582227/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.649602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ca7e103b664fe8bba10415dc767bf7, user_id=50dbee1565814083b1af91b3d1582227, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.652331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:45.663536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 928/1844] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:45 2022] PUT /identity/v3/projects/89ca7e103b664fe8bba10415dc767bf7/users/50dbee1565814083b1af91b3d1582227/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:45.673742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ecd6-75d5-4e6e-8064-8701a067aa39 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.674037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ecd6-75d5-4e6e-8064-8701a067aa39 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.674456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ecd6-75d5-4e6e-8064-8701a067aa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.700876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ba96ecd6-75d5-4e6e-8064-8701a067aa39 None None] MFA Rules not processed for user `50dbee1565814083b1af91b3d1582227`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:45.702071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ba96ecd6-75d5-4e6e-8064-8701a067aa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:45.770892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 917/1845] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:45.782654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:45.784414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:45.972758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695bbaa-55cb-403a-b1a7-26891cde17f8 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:45.977082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:45.977765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:45.978052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:45.978364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:45.988741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:45.990796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:46.000369 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 929/1846] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:46.214139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96ecd6-75d5-4e6e-8064-8701a067aa39 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:46.224179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96ecd6-75d5-4e6e-8064-8701a067aa39 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:46.228040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:46.228840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:46.229121 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:46.229392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:46.292413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:46.294348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:46.306665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 918/1847] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:46.319199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:47.855370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:47.865125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf726f-26e4-4fed-9482-7e91a08e7979 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:47.869506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:47.870382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:47.870662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:47.871107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:47.930963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:47.932503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:47.944255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 930/1848] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:48.511124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:48.716997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:49.694573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:49.760843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5eda83-34ec-4179-9abf-f2ceccffafb3 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:49.765729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:49.766666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:49.766943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:49.767211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] PATH_INFO: `/v3/users/a0916560b4f6454495eae2c804e9923b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:49.768383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0916560b4f6454495eae2c804e9923b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:49.771603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:50.104641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0916560b4f6454495eae2c804e9923b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:50.105429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0916560b4f6454495eae2c804e9923b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:50.114346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0916560b4f6454495eae2c804e9923b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:50.197312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:50.206201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241a119-1881-4cf7-a799-01b0eee62248 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:50.210032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:50.210773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:50.211429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:50.211899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:50.214067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:50.258337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0916560b4f6454495eae2c804e9923b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:50.271245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c40e8ebc-0ca2-460a-879e-648d73c7e6b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0916560b4f6454495eae2c804e9923b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:50.277304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 919/1849] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:49 2022] DELETE /identity/v3/users/a0916560b4f6454495eae2c804e9923b => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:50.281741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:50.282101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:50.282448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:50.296588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:50.298199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:50.300678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:50.301898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:50.308454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 931/1850] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:56:50.362667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 920/1851] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:50.373407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:50.374848 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:50.430312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972875d3-947e-4477-a681-f3256a949e75 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:50.478084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:50.478842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:50.479148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:50.479425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:50.489649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:50.491205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:50.500933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 932/1852] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:50.984793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:50.986266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:50.998634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fb1ec-29bd-4b8b-81b6-6b12e9395398 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:51.003943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:51.004929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:51.005239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:51.005450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] PATH_INFO: `/v3/projects/a81d0a7fbd5742059a7de18a6b09d316` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:51.006788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a81d0a7fbd5742059a7de18a6b09d316)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:51.009003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:51.156078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a81d0a7fbd5742059a7de18a6b09d316', 'request_id': 'req-6e311df6-2c7c-4f2b-afa2-097d7d88b295'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.162071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a81d0a7fbd5742059a7de18a6b09d316', 'request_id': 'req-6e311df6-2c7c-4f2b-afa2-097d7d88b295'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.269880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a81d0a7fbd5742059a7de18a6b09d316', 'request_id': 'req-6e311df6-2c7c-4f2b-afa2-097d7d88b295'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.285983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] The token cache is being invalidate because project a81d0a7fbd5742059a7de18a6b09d316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:51.285983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.285983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.286769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 921/1853] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:50 2022] DELETE /identity/v3/projects/a81d0a7fbd5742059a7de18a6b09d316 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:51.290692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:51.300422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10430ca8-0750-4a45-ae04-a5137b90c619 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:51.359356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:51.360402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:51.360761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:51.361115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] PATH_INFO: `/v3/users/50dbee1565814083b1af91b3d1582227` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:51.362527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50dbee1565814083b1af91b3d1582227)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:51.364561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:51.574894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50dbee1565814083b1af91b3d1582227'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.575845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50dbee1565814083b1af91b3d1582227'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.578336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50dbee1565814083b1af91b3d1582227'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.590764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50dbee1565814083b1af91b3d1582227'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.697865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50dbee1565814083b1af91b3d1582227'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:51.714069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 933/1854] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:51 2022] DELETE /identity/v3/users/50dbee1565814083b1af91b3d1582227 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:52.180645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:52.183288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:52.199287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e311df6-2c7c-4f2b-afa2-097d7d88b295 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:52.263132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:52.264035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.264323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.264605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] PATH_INFO: `/v3/projects/89ca7e103b664fe8bba10415dc767bf7` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.265870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ca7e103b664fe8bba10415dc767bf7)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:52.267876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:52.449044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:52.450521 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:52.463675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9ce7-9d1f-48f0-9b4d-f46aa9f5459c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:52.468628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:52.469626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.470010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.470414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] PATH_INFO: `/v3/users/c83a81bc7e6749239b654f0b1992aab8` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.472568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83a81bc7e6749239b654f0b1992aab8)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:52.480761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:52.492209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ca7e103b664fe8bba10415dc767bf7', 'request_id': 'req-efddc2c7-51ca-4589-ba33-33ce26bf5157'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.499696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ca7e103b664fe8bba10415dc767bf7', 'request_id': 'req-efddc2c7-51ca-4589-ba33-33ce26bf5157'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.610741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ca7e103b664fe8bba10415dc767bf7', 'request_id': 'req-efddc2c7-51ca-4589-ba33-33ce26bf5157'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.621497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] The token cache is being invalidate because project 89ca7e103b664fe8bba10415dc767bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:52.621783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.626538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-efddc2c7-51ca-4589-ba33-33ce26bf5157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.626538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 922/1855] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:52 2022] DELETE /identity/v3/projects/89ca7e103b664fe8bba10415dc767bf7 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:52.632078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166b7b9-7814-4577-bf68-553d18bd6216 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.632363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166b7b9-7814-4577-bf68-553d18bd6216 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.632639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166b7b9-7814-4577-bf68-553d18bd6216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.720348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c166b7b9-7814-4577-bf68-553d18bd6216 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:52.721832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c166b7b9-7814-4577-bf68-553d18bd6216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:52.763091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83a81bc7e6749239b654f0b1992aab8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.763796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83a81bc7e6749239b654f0b1992aab8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.764360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83a81bc7e6749239b654f0b1992aab8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.777072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83a81bc7e6749239b654f0b1992aab8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.784492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 923/1856] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:52.793249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4d76b-c9c6-4c90-b9ca-cc96a8ad5d1f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.793531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4d76b-c9c6-4c90-b9ca-cc96a8ad5d1f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.793807 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4d76b-c9c6-4c90-b9ca-cc96a8ad5d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.812544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5ea4d76b-c9c6-4c90-b9ca-cc96a8ad5d1f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:52.813744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5ea4d76b-c9c6-4c90-b9ca-cc96a8ad5d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:52.825149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 924/1857] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:52.835382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4d76b-c9c6-4c90-b9ca-cc96a8ad5d1f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:52.844408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4d76b-c9c6-4c90-b9ca-cc96a8ad5d1f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:52.849390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:52.850053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.850426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.850716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.851509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:52.851822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:52.853489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:52.865695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 925/1858] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:56:52.869719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:52.880099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977cced-4288-4b0c-9d2b-fad5ad61a486 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:52.885088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:52.885738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.886025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.886588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.887275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:52.889197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:52.896833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b4624a5a-a760-40f6-82e7-ff3f71c0e770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83a81bc7e6749239b654f0b1992aab8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:52.910695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 934/1859] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:52 2022] DELETE /identity/v3/users/c83a81bc7e6749239b654f0b1992aab8 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:52.915053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:52.915366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:52.915647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:52.934852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:52.939312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.000876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 926/1860] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:52 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:53.002145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.006566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 935/1861] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:53.008057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:53.016460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.018439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.068051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5e96a-e86b-4558-9da3-ce9a7b3843aa admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.073162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.073848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.074364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.074782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.075627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.077683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.078402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6a980-9b62-4559-9aff-9bb0df0ccf73 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.120531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:53.130775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.131553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.131918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.132269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.142065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.144118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.154737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 927/1862] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:53 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:53.159006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.162370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 936/1863] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:53.169827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934b63-0e82-43e9-990b-657796dea150 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.176719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.177480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.177833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.178337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.179142 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.180909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.194231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 928/1864] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:53.200475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.211199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9f064-7e67-4a4a-ba2a-47d976a2841a service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.216058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bdc5d-215d-4315-b0c8-f9daf76394ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.216731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bdc5d-215d-4315-b0c8-f9daf76394ed admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.217012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bdc5d-215d-4315-b0c8-f9daf76394ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.217278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bdc5d-215d-4315-b0c8-f9daf76394ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.217998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bdc5d-215d-4315-b0c8-f9daf76394ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.220033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bdc5d-215d-4315-b0c8-f9daf76394ed admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.235368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 937/1865] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:53.236072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.249855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087a1c1-5a7c-4f08-be9e-80bf109edd16 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.256670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.257500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.258970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.259380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] PATH_INFO: `/v3/projects/e8f113ad67d148e0adbd3a1e65d99758/users/b73bc10c8aa14bffacb044854de2ab86/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.275987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f113ad67d148e0adbd3a1e65d99758, user_id=b73bc10c8aa14bffacb044854de2ab86, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.281966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.424365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 929/1866] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:53 2022] PUT /identity/v3/projects/e8f113ad67d148e0adbd3a1e65d99758/users/b73bc10c8aa14bffacb044854de2ab86/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:53.442106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.442580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.443009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.473812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] MFA Rules not processed for user `b73bc10c8aa14bffacb044854de2ab86`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:53.475264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.542745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 938/1867] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:53.547404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.624909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf25eb-aacf-49d5-9940-a3c336949165 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.631280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.631922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.632212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.632496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.633676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.645031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.826149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 930/1868] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:53 2022] POST /identity/v3/projects => generated 387 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:53.828158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.829678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:53.833819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:53.855896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d65ff-6ca9-4727-a942-a7631b777c4e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.903013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e9dd-942a-4875-8186-2e07683f8e9a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:53.908022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.908849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.909131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.909407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.910338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.912418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:53.949675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:53.950988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:53.951645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:53.952229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] PATH_INFO: `/v3/projects/6880552302ad4736b37c41a389584c65` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:53.954002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6880552302ad4736b37c41a389584c65)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:53.957493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.006205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 931/1869] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:53 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:54.010565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:54.021732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cd356-805a-4722-9abc-d3fb9bbcad79 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.026997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.027668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.027948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.028221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.028940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.030658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.042179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 932/1870] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:54.046135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:54.056544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d02b2-ac3c-4867-bb61-02bbd9a83f5a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.061254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.062120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.062430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.062703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] PATH_INFO: `/v3/projects/9206945765a04b0fa8f7a9ff4e654b43/users/0b955d9d2b0d4c99bf8cd221d11fca5e/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.079025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9206945765a04b0fa8f7a9ff4e654b43, user_id=0b955d9d2b0d4c99bf8cd221d11fca5e, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.082256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.234103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 933/1871] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:54 2022] PUT /identity/v3/projects/9206945765a04b0fa8f7a9ff4e654b43/users/0b955d9d2b0d4c99bf8cd221d11fca5e/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:54.238077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:54.245523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6880552302ad4736b37c41a389584c65', 'request_id': 'req-bf231119-ff53-4462-91bf-eab21a2e3ac6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.304472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279863a2-3e22-4167-9177-ea069809d41f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.361835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.362542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.362867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.363144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.363870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.366055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.375692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 934/1872] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:54.386909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:54.390891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdd9a7-f292-4f89-85e9-4e17321f3a3d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.397098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4233967-9641-4a45-ab69-a6aae0ddd5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.398153 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4233967-9641-4a45-ab69-a6aae0ddd5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.398612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4233967-9641-4a45-ab69-a6aae0ddd5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.398948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4233967-9641-4a45-ab69-a6aae0ddd5f1 admin admin] PATH_INFO: `/v3/projects/9206945765a04b0fa8f7a9ff4e654b43/users/0b955d9d2b0d4c99bf8cd221d11fca5e/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.401918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4233967-9641-4a45-ab69-a6aae0ddd5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9206945765a04b0fa8f7a9ff4e654b43, user_id=0b955d9d2b0d4c99bf8cd221d11fca5e, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.405190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4233967-9641-4a45-ab69-a6aae0ddd5f1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.415700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 935/1873] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:54 2022] PUT /identity/v3/projects/9206945765a04b0fa8f7a9ff4e654b43/users/0b955d9d2b0d4c99bf8cd221d11fca5e/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:54.427640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.428752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.428752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.433423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6880552302ad4736b37c41a389584c65', 'request_id': 'req-bf231119-ff53-4462-91bf-eab21a2e3ac6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.442344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6880552302ad4736b37c41a389584c65', 'request_id': 'req-bf231119-ff53-4462-91bf-eab21a2e3ac6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.456220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] The token cache is being invalidate because project 6880552302ad4736b37c41a389584c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:54.456891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.457839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] MFA Rules not processed for user `0b955d9d2b0d4c99bf8cd221d11fca5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:54.458674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.459102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:54.460858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 939/1874] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:53 2022] DELETE /identity/v3/projects/6880552302ad4736b37c41a389584c65 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:54.464942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:54.467229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:54.487270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231119-ff53-4462-91bf-eab21a2e3ac6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.542519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 936/1875] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:54.554720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:54.556344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:54.566793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.567757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.568109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.568387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] PATH_INFO: `/v3/users/4c1e46c391414137bc4de6aea166dfc1` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.570055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1e46c391414137bc4de6aea166dfc1)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.572343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.615177 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c7f96-84de-4514-a8ac-aa6ae0b3c0e5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.619770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.620494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.621167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.621448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.631590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.633102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.642890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 937/1876] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:54.779296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:54.780811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:54.795399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0410a0b-2b45-4342-95ae-85ac5ebb30f9 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:54.801314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:54.802199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:54.802492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:54.802762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] PATH_INFO: `/v3/users/b73bc10c8aa14bffacb044854de2ab86` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:54.803958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b73bc10c8aa14bffacb044854de2ab86)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:54.805880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:54.881951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1e46c391414137bc4de6aea166dfc1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.882542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1e46c391414137bc4de6aea166dfc1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.883155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1e46c391414137bc4de6aea166dfc1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.894981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1e46c391414137bc4de6aea166dfc1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.964896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-456a0feb-a9e0-4aca-9cc2-064628fe7984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1e46c391414137bc4de6aea166dfc1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:54.980146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 940/1877] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:54 2022] DELETE /identity/v3/users/4c1e46c391414137bc4de6aea166dfc1 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:55.058572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b73bc10c8aa14bffacb044854de2ab86'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.059256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b73bc10c8aa14bffacb044854de2ab86'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.066644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b73bc10c8aa14bffacb044854de2ab86'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.177840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b73bc10c8aa14bffacb044854de2ab86'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.195292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b73bc10c8aa14bffacb044854de2ab86'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.201996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 938/1878] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:54 2022] DELETE /identity/v3/users/b73bc10c8aa14bffacb044854de2ab86 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:55.206871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:55.207164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:55.207447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:55.227227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:55.228954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:55.293428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 941/1879] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:55.304251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:55.305764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:55.375567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f0a15-755c-4c8b-9ee1-0aae207feb92 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:55.380076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:55.380791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:55.381091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:55.381352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:55.402320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:55.402320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:55.404804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:55.406327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:55.408714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 939/1880] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:55.419016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8289cd8-f3d8-466f-b758-aee9596fb570 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:55.425219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:55.426517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:55.426871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:55.427290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] PATH_INFO: `/v3/users/18630c7f125040718fbcc651348941f6` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:55.428593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=18630c7f125040718fbcc651348941f6)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:55.430945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:55.446939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:55.449209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:55.467381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bd8e7-aef5-4aa9-83fd-429288867e56 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:55.476741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:55.476741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:55.476741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:55.477089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] PATH_INFO: `/v3/projects/8f6b9ad34cbe4993aa929df84b7968e8` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:55.478260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6b9ad34cbe4993aa929df84b7968e8)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:55.480199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:55.801611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18630c7f125040718fbcc651348941f6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.803694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18630c7f125040718fbcc651348941f6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.804672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18630c7f125040718fbcc651348941f6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.807477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6b9ad34cbe4993aa929df84b7968e8', 'request_id': 'req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.815887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6b9ad34cbe4993aa929df84b7968e8', 'request_id': 'req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.821154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18630c7f125040718fbcc651348941f6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.920572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6b9ad34cbe4993aa929df84b7968e8', 'request_id': 'req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.933254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] The token cache is being invalidate because project 8f6b9ad34cbe4993aa929df84b7968e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:55.933524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.934360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:55.936161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 940/1881] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:55 2022] DELETE /identity/v3/projects/8f6b9ad34cbe4993aa929df84b7968e8 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:55.936719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:55.938080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.006132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4697caa-fef9-4b4c-a192-eef7ddc9b0d4 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.011089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.012003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.012268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.012543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] PATH_INFO: `/v3/projects/e8f113ad67d148e0adbd3a1e65d99758` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.013737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f113ad67d148e0adbd3a1e65d99758)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.015708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.061765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-41229780-3058-4da4-a75a-63836d16a4bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18630c7f125040718fbcc651348941f6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.075710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 942/1882] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:55 2022] DELETE /identity/v3/users/18630c7f125040718fbcc651348941f6 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:56.076981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17d2c-dec6-4e3b-a88c-62327b353dba None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.077729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17d2c-dec6-4e3b-a88c-62327b353dba None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.077729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17d2c-dec6-4e3b-a88c-62327b353dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.100504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-dbf17d2c-dec6-4e3b-a88c-62327b353dba None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:56.100504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dbf17d2c-dec6-4e3b-a88c-62327b353dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.158843 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 943/1883] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:56.159908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cccc-362a-4628-a54d-4b9dfafe55f2 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.160201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cccc-362a-4628-a54d-4b9dfafe55f2 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.160669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cccc-362a-4628-a54d-4b9dfafe55f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.183917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e8b6cccc-362a-4628-a54d-4b9dfafe55f2 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:56.186248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e8b6cccc-362a-4628-a54d-4b9dfafe55f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.188178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:56.190426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f113ad67d148e0adbd3a1e65d99758', 'request_id': 'req-7da0a023-0c7d-4b42-8a17-797b092e2c65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.198444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f113ad67d148e0adbd3a1e65d99758', 'request_id': 'req-7da0a023-0c7d-4b42-8a17-797b092e2c65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.216006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:56.257650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 944/1884] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:56.259015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.259305 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.259579 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.278825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:56.280035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.289059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 945/1885] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:56.289849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.291086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.331069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f113ad67d148e0adbd3a1e65d99758', 'request_id': 'req-7da0a023-0c7d-4b42-8a17-797b092e2c65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.342261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] The token cache is being invalidate because project e8f113ad67d148e0adbd3a1e65d99758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:56.342557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.343213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.344967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 941/1886] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:55 2022] DELETE /identity/v3/projects/e8f113ad67d148e0adbd3a1e65d99758 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:56.345755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.347126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.347725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cc4ce-df2d-4dd2-979c-e70ea948b032 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.351572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.352273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.352564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.352832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.354742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.359848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0a023-0c7d-4b42-8a17-797b092e2c65 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.364485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.365137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.365423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.365703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.366479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:56.366679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.366792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.368116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.368396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.377948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 946/1887] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:56.378783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.380092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.382548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 942/1888] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:56:56.386935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.398077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d32b0-1b3d-43a6-ae12-0df5ada18a28 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.398338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bec9d9-1f5c-4be1-ac1c-1fbe90031159 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.402755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.403202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.403539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.403834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.404113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.404221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.404448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.404791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] PATH_INFO: `/v3/users/0b955d9d2b0d4c99bf8cd221d11fca5e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.404891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.406034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b955d9d2b0d4c99bf8cd221d11fca5e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.406859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.412498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.588639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 943/1889] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:56 2022] POST /identity/v3/projects => generated 369 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:56.591977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.684149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d15b3-ebb5-45ba-94e7-75e04326ee08 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.692491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.693513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.694559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.694863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.695720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.697695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.777724 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b955d9d2b0d4c99bf8cd221d11fca5e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.778772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b955d9d2b0d4c99bf8cd221d11fca5e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.779463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b955d9d2b0d4c99bf8cd221d11fca5e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.793495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 944/1890] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:56 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:56.794669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.798651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:56.799870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b955d9d2b0d4c99bf8cd221d11fca5e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.810526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db09ad-2abc-4be7-8da5-7de8a84b8775 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.817582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.818299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.818586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.818866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.819610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.821237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.836018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 945/1891] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:56.846197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.856353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a978b4-2227-4250-9b2d-9955babf74c1 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.863241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.864273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.864599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.864882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.865647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.868420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.881526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 946/1892] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:56.886261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.896726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315eb6-36d3-4a39-8136-7877371a72a8 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:56.901471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b955d9d2b0d4c99bf8cd221d11fca5e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:56.902569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c91447-3bd9-4269-a383-bb08b2d47d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:56.903468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c91447-3bd9-4269-a383-bb08b2d47d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:56.904700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c91447-3bd9-4269-a383-bb08b2d47d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:56.904700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c91447-3bd9-4269-a383-bb08b2d47d49 admin admin] PATH_INFO: `/v3/projects/c90d566db2284741b688901399b49e56/users/c87aa8ce243f4c1e8ed1eb48cf324cd5/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:56.918013 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 947/1893] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:56 2022] DELETE /identity/v3/users/0b955d9d2b0d4c99bf8cd221d11fca5e => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:56.927880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c91447-3bd9-4269-a383-bb08b2d47d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90d566db2284741b688901399b49e56, user_id=c87aa8ce243f4c1e8ed1eb48cf324cd5, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:56.930861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c91447-3bd9-4269-a383-bb08b2d47d49 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:56.961678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:56.965112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:57.042873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 947/1894] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:56 2022] PUT /identity/v3/projects/c90d566db2284741b688901399b49e56/users/c87aa8ce243f4c1e8ed1eb48cf324cd5/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:57.051774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ea1ee-3376-4de6-90d5-d28a2650f7ac None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.052747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ea1ee-3376-4de6-90d5-d28a2650f7ac None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.053121 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ea1ee-3376-4de6-90d5-d28a2650f7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.059292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647c1b8-ee25-4faa-bd29-8383964b0bbe admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:57.093616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-865ea1ee-3376-4de6-90d5-d28a2650f7ac None None] MFA Rules not processed for user `c87aa8ce243f4c1e8ed1eb48cf324cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:57.095635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-865ea1ee-3376-4de6-90d5-d28a2650f7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:57.144919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:57.146035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.146672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.147051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] PATH_INFO: `/v3/projects/c5f62642d18d40329dde836786760296` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.149031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f62642d18d40329dde836786760296)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:57.152273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:57.168975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 948/1895] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:57.175639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ea1ee-3376-4de6-90d5-d28a2650f7ac None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:57.186920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ea1ee-3376-4de6-90d5-d28a2650f7ac None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:57.196222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:57.197033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.197548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.198045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.198936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:57.201039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:57.287638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f62642d18d40329dde836786760296', 'request_id': 'req-8c987a92-de5a-400a-ba04-d6d1e81ce69f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.382045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 949/1896] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:57 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:57.383197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:57.401090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f62642d18d40329dde836786760296', 'request_id': 'req-8c987a92-de5a-400a-ba04-d6d1e81ce69f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.408950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f62642d18d40329dde836786760296', 'request_id': 'req-8c987a92-de5a-400a-ba04-d6d1e81ce69f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.426269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] The token cache is being invalidate because project c5f62642d18d40329dde836786760296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:57.426515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.427740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.431545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 948/1897] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:56 2022] DELETE /identity/v3/projects/c5f62642d18d40329dde836786760296 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:57.441044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:57.443081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:57.474523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a17bc-cf69-4dff-bc64-d095d2d98b7a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:57.479779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:57.480464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.480749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.481051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.481878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:57.483872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:57.520187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c987a92-de5a-400a-ba04-d6d1e81ce69f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:57.525213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:57.526133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.526516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.526801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] PATH_INFO: `/v3/users/a2c111337a1e4b92ae8b7cdd06c04946` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.528027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c111337a1e4b92ae8b7cdd06c04946)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:57.529989 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:57.568853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 950/1898] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:57 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:57.569891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:57.571650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:57.584325 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e995e-08de-473c-aa75-788574779d21 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:57.589254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:57.590132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.590707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.590903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] PATH_INFO: `/v3/projects/9206945765a04b0fa8f7a9ff4e654b43` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.592127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9206945765a04b0fa8f7a9ff4e654b43)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:57.594321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:57.748724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9206945765a04b0fa8f7a9ff4e654b43', 'request_id': 'req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.749049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c111337a1e4b92ae8b7cdd06c04946'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.749789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c111337a1e4b92ae8b7cdd06c04946'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.755973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c111337a1e4b92ae8b7cdd06c04946'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.757179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9206945765a04b0fa8f7a9ff4e654b43', 'request_id': 'req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.769791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c111337a1e4b92ae8b7cdd06c04946'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.838076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9206945765a04b0fa8f7a9ff4e654b43', 'request_id': 'req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.850393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] The token cache is being invalidate because project 9206945765a04b0fa8f7a9ff4e654b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:57.850760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.851512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.853144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 951/1899] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:57 2022] DELETE /identity/v3/projects/9206945765a04b0fa8f7a9ff4e654b43 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:57.857192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:57.858532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:57.918022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-932651f9-fc26-486c-a751-7165eac1e69e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c111337a1e4b92ae8b7cdd06c04946'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:57.926914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a1e1-b6ee-4ea8-9110-bdb98e5af08c admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:57.931955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:57.932487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 949/1900] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:57 2022] DELETE /identity/v3/users/a2c111337a1e4b92ae8b7cdd06c04946 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:57.932709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.932958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.933504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.934332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:57.935031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffba9b-a183-433d-a5de-83bcdeebe358 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:57.935663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffba9b-a183-433d-a5de-83bcdeebe358 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:57.936678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffba9b-a183-433d-a5de-83bcdeebe358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:57.939344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:57.954543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 952/1901] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:57.961209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:57.968341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-31ffba9b-a183-433d-a5de-83bcdeebe358 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:57.971009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-31ffba9b-a183-433d-a5de-83bcdeebe358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:57.977151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dea74f-9ea6-4145-8bb1-b8760a011959 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.037771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 950/1902] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:58.042476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.043468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.043933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.044045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] PATH_INFO: `/v3/projects/711d47f41aaf405b80d9d06c856a0e49/users/2de9380237254ed1ac8de91f1607c398/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.045383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fc8dd-abc3-44c2-a87a-ee8577b62de2 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.045681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fc8dd-abc3-44c2-a87a-ee8577b62de2 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.045961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fc8dd-abc3-44c2-a87a-ee8577b62de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.059925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=711d47f41aaf405b80d9d06c856a0e49, user_id=2de9380237254ed1ac8de91f1607c398, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.064498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.066056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3e5fc8dd-abc3-44c2-a87a-ee8577b62de2 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:58.067318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3e5fc8dd-abc3-44c2-a87a-ee8577b62de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.075400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 951/1903] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:58.080429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fc8dd-abc3-44c2-a87a-ee8577b62de2 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.089647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fc8dd-abc3-44c2-a87a-ee8577b62de2 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.094729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.095406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.095688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.095966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.096731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:56:58.097124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.099294 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.118658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 952/1904] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:56:58.120177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.132929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7dd3-7ef9-4532-960c-e61f7d4baef4 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.138429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.139104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.139388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.139734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.140399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.142988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.193842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 953/1905] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:57 2022] PUT /identity/v3/projects/711d47f41aaf405b80d9d06c856a0e49/users/2de9380237254ed1ac8de91f1607c398/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:58.199284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.274037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 953/1906] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:58 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:58.278221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.300300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77010f71-10a7-4320-b8b6-1c1d09e479e2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.346342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35ad69-4c76-43df-9f3b-6d8494e77fc6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.351288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.351970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.352261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.352537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.353233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.355511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.372841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.373580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.373894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.374156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.374958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.376917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.390216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 954/1907] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:56:58.396973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.404081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b50f28-17d3-4639-a78c-07144e234028 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.410023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad0098-f46a-4ef0-8ae8-6fe8e330e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.410972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad0098-f46a-4ef0-8ae8-6fe8e330e486 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.411311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad0098-f46a-4ef0-8ae8-6fe8e330e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.411593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad0098-f46a-4ef0-8ae8-6fe8e330e486 admin admin] PATH_INFO: `/v3/projects/711d47f41aaf405b80d9d06c856a0e49/users/2de9380237254ed1ac8de91f1607c398/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.413597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad0098-f46a-4ef0-8ae8-6fe8e330e486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=711d47f41aaf405b80d9d06c856a0e49, user_id=2de9380237254ed1ac8de91f1607c398, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.416302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad0098-f46a-4ef0-8ae8-6fe8e330e486 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.427623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 955/1908] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:58 2022] PUT /identity/v3/projects/711d47f41aaf405b80d9d06c856a0e49/users/2de9380237254ed1ac8de91f1607c398/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:56:58.437229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce09518-8148-4a5a-b744-af59992f34a3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.437599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce09518-8148-4a5a-b744-af59992f34a3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.437979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce09518-8148-4a5a-b744-af59992f34a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.470016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-fce09518-8148-4a5a-b744-af59992f34a3 None None] MFA Rules not processed for user `2de9380237254ed1ac8de91f1607c398`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:58.470190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 954/1909] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:58 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:56:58.470303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fce09518-8148-4a5a-b744-af59992f34a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.472371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.481727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.507483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39844-0d19-4c90-be63-e355ce1602f2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.567104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 956/1910] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:56:58.567900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce09518-8148-4a5a-b744-af59992f34a3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.595604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.596679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.597227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.597578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] PATH_INFO: `/v3/projects/fac25ade60bb48b5832b21064903566c` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.599195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fac25ade60bb48b5832b21064903566c)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.603251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:58.603774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.655370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce09518-8148-4a5a-b744-af59992f34a3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.660359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.661047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.661359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.661641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.662426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.664068 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.677189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 957/1911] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:58.677189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.677189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.704881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:56:58.735437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100f4b-20cf-4797-8ac0-47ee0c582dd7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.739014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf06f8-1d81-48a1-a11d-a1f9fbe55b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.739655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf06f8-1d81-48a1-a11d-a1f9fbe55b88 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.739945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf06f8-1d81-48a1-a11d-a1f9fbe55b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.740221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf06f8-1d81-48a1-a11d-a1f9fbe55b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.749733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf06f8-1d81-48a1-a11d-a1f9fbe55b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.751449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf06f8-1d81-48a1-a11d-a1f9fbe55b88 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.761917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 958/1912] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:58.763304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250bb43-0c77-41cd-8ef1-6d9ef9cfb59e None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.763643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250bb43-0c77-41cd-8ef1-6d9ef9cfb59e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.763935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250bb43-0c77-41cd-8ef1-6d9ef9cfb59e None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.765226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 959/1913] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:56:58.765954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250bb43-0c77-41cd-8ef1-6d9ef9cfb59e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.771678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fac25ade60bb48b5832b21064903566c', 'request_id': 'req-f62246d6-38b9-4423-9928-0eb57304bcdc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.836370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250bb43-0c77-41cd-8ef1-6d9ef9cfb59e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.840951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.841612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.841939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.842252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.842974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.845268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.848835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fac25ade60bb48b5832b21064903566c', 'request_id': 'req-f62246d6-38b9-4423-9928-0eb57304bcdc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.855092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fac25ade60bb48b5832b21064903566c', 'request_id': 'req-f62246d6-38b9-4423-9928-0eb57304bcdc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.860625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 960/1914] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:56:58.860625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.860625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.867829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] The token cache is being invalidate because project fac25ade60bb48b5832b21064903566c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:56:58.868119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.868786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f62246d6-38b9-4423-9928-0eb57304bcdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:58.870389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 955/1915] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:58 2022] DELETE /identity/v3/projects/fac25ade60bb48b5832b21064903566c => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:58.871790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.872075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.872349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.874147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e80a7-23b2-492d-858f-8c4f52dd7466 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.879752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.880699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.881055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.881396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] PATH_INFO: `/v3/users/5fdf869a706a445d950ee8f59595ad4a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.882772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fdf869a706a445d950ee8f59595ad4a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.885183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:58.891942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] MFA Rules not processed for user `6998e874e8264977a7dc412436162a02`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:58.893201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.956880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 956/1916] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:56:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:58.957719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:58.959012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:58.972930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1dab-3143-4e26-8d63-4bd026361cbc None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:58.981208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:58.981208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:58.981208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:58.981208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] PATH_INFO: `/v3/projects/c239e6e2282943babee45377c94f848f/users/50b25b11d00341b5ac5d3e25263eb17f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:58.995339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c239e6e2282943babee45377c94f848f, user_id=50b25b11d00341b5ac5d3e25263eb17f, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:58.998101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.105604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fdf869a706a445d950ee8f59595ad4a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.106293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fdf869a706a445d950ee8f59595ad4a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.113961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fdf869a706a445d950ee8f59595ad4a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.226583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 957/1917] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:56:58 2022] PUT /identity/v3/projects/c239e6e2282943babee45377c94f848f/users/50b25b11d00341b5ac5d3e25263eb17f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:59.227554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.229194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.238376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fdf869a706a445d950ee8f59595ad4a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.253900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fdf869a706a445d950ee8f59595ad4a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.264265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 961/1918] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:58 2022] DELETE /identity/v3/users/5fdf869a706a445d950ee8f59595ad4a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:59.264265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.264265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.301529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043950a-170b-414e-b329-6c69ee48dc9d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.351510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcfac0-d323-4d7e-9648-586c264765d8 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.356908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672675ad-aa19-459e-9a0f-458486932b6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.357296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672675ad-aa19-459e-9a0f-458486932b6b service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.357584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672675ad-aa19-459e-9a0f-458486932b6b service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.357864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672675ad-aa19-459e-9a0f-458486932b6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.359731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-672675ad-aa19-459e-9a0f-458486932b6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.362967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.364262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.364530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.364815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] PATH_INFO: `/v3/users/2b90a098170a4e0a89de44e513301871` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.366446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b90a098170a4e0a89de44e513301871)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.368439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.373646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672675ad-aa19-459e-9a0f-458486932b6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.375856 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672675ad-aa19-459e-9a0f-458486932b6b service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.387031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 962/1919] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:56:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:56:59.387979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.388264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.388547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.443662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] MFA Rules not processed for user `50b25b11d00341b5ac5d3e25263eb17f`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:56:59.443662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.494553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 963/1920] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:56:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:56:59.498817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.500776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.513936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770e126-928a-4323-baa2-5add3bcc7d68 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.518794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.519456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.519736 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.520015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.520709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.522668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.741831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 964/1921] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:56:59 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:56:59.743132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.747730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.751974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b90a098170a4e0a89de44e513301871'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.754221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b90a098170a4e0a89de44e513301871'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.754221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b90a098170a4e0a89de44e513301871'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.769805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b90a098170a4e0a89de44e513301871'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.831538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c855b9-a601-41b8-8f3b-0bc9463758d6 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.836109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.836968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.837251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.837528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] PATH_INFO: `/v3/projects/dbb7af9fb2864248800c775d2d0d174e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.838814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb7af9fb2864248800c775d2d0d174e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.840723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:56:59.870514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b90a098170a4e0a89de44e513301871'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:56:59.883646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 958/1922] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:56:59 2022] DELETE /identity/v3/users/2b90a098170a4e0a89de44e513301871 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:56:59.884734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:56:59.886275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:56:59.954432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061f0e-ba1c-420a-9200-cb2a3e7e6427 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:56:59.959792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:56:59.960557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:56:59.960998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:56:59.961270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:56:59.961906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:56:59.963905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.032199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb7af9fb2864248800c775d2d0d174e', 'request_id': 'req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.038304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb7af9fb2864248800c775d2d0d174e', 'request_id': 'req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.100336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb7af9fb2864248800c775d2d0d174e', 'request_id': 'req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.107559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 959/1923] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:56:59 2022] POST /identity/v3/users => generated 299 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:00.116209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.125754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] The token cache is being invalidate because project dbb7af9fb2864248800c775d2d0d174e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:00.126449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.127303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4c97dcd2-a5e0-449d-9d8d-aa4c61cca4a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.129757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 965/1924] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:56:59 2022] DELETE /identity/v3/projects/dbb7af9fb2864248800c775d2d0d174e => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:00.130803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770dee9-2275-48ca-87f2-dc03bc18ba65 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.134854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d223e51-d427-4f70-9340-815f0aad878e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.135197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d223e51-d427-4f70-9340-815f0aad878e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.135483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d223e51-d427-4f70-9340-815f0aad878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.162607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1d223e51-d427-4f70-9340-815f0aad878e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:00.163994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1d223e51-d427-4f70-9340-815f0aad878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.204309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.204911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.205253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.205591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.206429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.208462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.222595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 960/1925] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:00.224105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.225940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.233799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 966/1926] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:00.240853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7773-1242-4092-9a66-b600e18cccb2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.244010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a0b6-2889-4373-a624-31ffa5528e24 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.244484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a0b6-2889-4373-a624-31ffa5528e24 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.244916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a0b6-2889-4373-a624-31ffa5528e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.247598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.248623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.248916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.249187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] PATH_INFO: `/v3/projects/e98e5c48a2de4d8cb291c3769b02718a/users/8cf121e25557465184db7520167d476f/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.265812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-deb2a0b6-2889-4373-a624-31ffa5528e24 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:00.267733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-deb2a0b6-2889-4373-a624-31ffa5528e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.267858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98e5c48a2de4d8cb291c3769b02718a, user_id=8cf121e25557465184db7520167d476f, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.270694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.276842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 967/1927] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:00.281444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2a0b6-2889-4373-a624-31ffa5528e24 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.291810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2a0b6-2889-4373-a624-31ffa5528e24 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.301628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.301628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.301628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.301628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.301952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:00.301952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.303219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.315785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 968/1928] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:57:00.320220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.330505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5f88e-a74b-4dfe-8ed3-420e7a106247 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.335225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.335883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.336165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.336442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.337130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.339077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.374715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 961/1929] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:00 2022] PUT /identity/v3/projects/e98e5c48a2de4d8cb291c3769b02718a/users/8cf121e25557465184db7520167d476f/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:00.378306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.440709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026284c3-edbd-4e9e-8d4c-5b8d19c254f9 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.445421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.446088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.446400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.446676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.447399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.449012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.460363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 962/1930] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:00.461053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.473232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.473410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 969/1931] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:00 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:00.474997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.479693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d687-c457-4f6a-92cb-354961aa8cb7 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.538720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.539532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.539813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.540104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] PATH_INFO: `/v3/projects/d780bf4ae9134800b33c940b76c7edd7` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.541270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d780bf4ae9134800b33c940b76c7edd7)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.542846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d430af-a091-4e53-a354-cf296a1fb6e2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.543284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.554747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.555741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.556028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.556309 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] PATH_INFO: `/v3/projects/e98e5c48a2de4d8cb291c3769b02718a/users/8cf121e25557465184db7520167d476f/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.558458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98e5c48a2de4d8cb291c3769b02718a, user_id=8cf121e25557465184db7520167d476f, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.561125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.571980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 970/1932] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:00 2022] PUT /identity/v3/projects/e98e5c48a2de4d8cb291c3769b02718a/users/8cf121e25557465184db7520167d476f/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:00.579246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.641523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426cbb3-741e-4889-a8a4-68ef5a8908e7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:00.646229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a69882-87b1-46ad-a416-37604669fd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:00.646909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69882-87b1-46ad-a416-37604669fd33 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.647220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69882-87b1-46ad-a416-37604669fd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.647499 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69882-87b1-46ad-a416-37604669fd33 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.648193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a69882-87b1-46ad-a416-37604669fd33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:00.652526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a69882-87b1-46ad-a416-37604669fd33 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:00.758918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd780bf4ae9134800b33c940b76c7edd7', 'request_id': 'req-fbeefb63-1a99-410f-b007-7466db723e2b'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.817941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 971/1933] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:00 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:00.817941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb45136-dd7a-4562-a630-84565e73d879 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.817941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb45136-dd7a-4562-a630-84565e73d879 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.817941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb45136-dd7a-4562-a630-84565e73d879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.845772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8fb45136-dd7a-4562-a630-84565e73d879 None None] MFA Rules not processed for user `8cf121e25557465184db7520167d476f`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:00.847050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8fb45136-dd7a-4562-a630-84565e73d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.893223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd780bf4ae9134800b33c940b76c7edd7', 'request_id': 'req-fbeefb63-1a99-410f-b007-7466db723e2b'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.906571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:00.910004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd780bf4ae9134800b33c940b76c7edd7', 'request_id': 'req-fbeefb63-1a99-410f-b007-7466db723e2b'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.924890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] The token cache is being invalidate because project d780bf4ae9134800b33c940b76c7edd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:00.925345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.926165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:00.928161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 963/1934] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:00 2022] DELETE /identity/v3/projects/d780bf4ae9134800b33c940b76c7edd7 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:00.937080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 972/1935] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:00.937290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:00.937484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae123f-fbc6-4103-a5e7-280d53086df5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:00.937484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae123f-fbc6-4103-a5e7-280d53086df5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:00.937484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae123f-fbc6-4103-a5e7-280d53086df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:00.939237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:00.958785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-80ae123f-fbc6-4103-a5e7-280d53086df5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:00.960375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-80ae123f-fbc6-4103-a5e7-280d53086df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.010165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:01.014042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeefb63-1a99-410f-b007-7466db723e2b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.019520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.020202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.021140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.021140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.021691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.023444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.041100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 964/1936] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:01.041100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.041100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.041897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 973/1937] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:01.043290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae123f-fbc6-4103-a5e7-280d53086df5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.063001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae123f-fbc6-4103-a5e7-280d53086df5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.063001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a444ec5-edd3-4060-a40e-7928f476ac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.063001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a444ec5-edd3-4060-a40e-7928f476ac74 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.063001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a444ec5-edd3-4060-a40e-7928f476ac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.063757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a444ec5-edd3-4060-a40e-7928f476ac74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.063948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a444ec5-edd3-4060-a40e-7928f476ac74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.066938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a444ec5-edd3-4060-a40e-7928f476ac74 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.088553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 974/1938] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:01.089905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4118c-59f2-41ae-b966-b6537ecd55d0 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.090227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4118c-59f2-41ae-b966-b6537ecd55d0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.090573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4118c-59f2-41ae-b966-b6537ecd55d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.111770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-80a4118c-59f2-41ae-b966-b6537ecd55d0 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:01.113138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-80a4118c-59f2-41ae-b966-b6537ecd55d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.126115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 975/1939] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:01.126115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4118c-59f2-41ae-b966-b6537ecd55d0 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.142368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4118c-59f2-41ae-b966-b6537ecd55d0 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.147349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072188-9ffe-4823-84ff-adf9a73df3dd admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.149371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab3b32-5096-4594-8b4e-432e9f59fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.150610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab3b32-5096-4594-8b4e-432e9f59fa48 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.151108 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.151251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab3b32-5096-4594-8b4e-432e9f59fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.151770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab3b32-5096-4594-8b4e-432e9f59fa48 admin admin] PATH_INFO: `/v3/projects/b79e365a60bf4723adf5ef861272ff48/users/d1c89c198a364b05998841b87b22c76d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.158054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.158054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.158054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.158054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.167649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab3b32-5096-4594-8b4e-432e9f59fa48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79e365a60bf4723adf5ef861272ff48, user_id=d1c89c198a364b05998841b87b22c76d, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.167908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.169358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.170357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab3b32-5096-4594-8b4e-432e9f59fa48 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.179813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 965/1940] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:01.180701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.191211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff0446-087e-4352-bfb0-61c75de86f4e service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.195900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.196609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.196895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.197170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.198019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:01.198393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.203725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.216364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 966/1941] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:01.218712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.228533 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf481-bc65-469b-a433-cda3ce1fda0b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.233059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.234274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.234562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.234839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.235609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.237627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.254013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 976/1942] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:01 2022] PUT /identity/v3/projects/b79e365a60bf4723adf5ef861272ff48/users/d1c89c198a364b05998841b87b22c76d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:01.266506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.266826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.267132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.293196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] MFA Rules not processed for user `d1c89c198a364b05998841b87b22c76d`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:01.294425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.348846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 967/1943] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:01 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:01.351251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.367836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 977/1944] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:01.380044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.381812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:01.428535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b946a30-20db-4620-96ef-7d31b9df2a60 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.433978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.434955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.435311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.435688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.436491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.439881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.475575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc8ffc-4c18-4ab6-8323-545952fcfb64 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.539732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.540391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.540672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.540946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.586384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 968/1945] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:01 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:01.589778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.601366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f9a67-5bcf-447a-a409-40e0852c3240 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.606789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.607713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.607969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.608414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.609055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.610665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.610839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.612403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.622679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 978/1946] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:01.629549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 969/1947] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:01.630854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.644270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c1d86-748f-47e6-af9a-a46cf7b73329 service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.649199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc91231-9395-4047-bca8-0db3c14e5196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.656501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc91231-9395-4047-bca8-0db3c14e5196 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.656501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc91231-9395-4047-bca8-0db3c14e5196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.656501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc91231-9395-4047-bca8-0db3c14e5196 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.656501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc91231-9395-4047-bca8-0db3c14e5196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.656501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc91231-9395-4047-bca8-0db3c14e5196 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.670217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 979/1948] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:01.674901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.693875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651162c-c16d-4352-97e3-5d1dd35190d2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:01.706054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:01.706054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.706054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.706054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] PATH_INFO: `/v3/projects/d8c2bc0c830b48b784c2057838100d5f/users/82b471aaac5a4793a370872a6206c917/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.716750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c2bc0c830b48b784c2057838100d5f, user_id=82b471aaac5a4793a370872a6206c917, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:01.720642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:01.901604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 970/1949] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:01 2022] PUT /identity/v3/projects/d8c2bc0c830b48b784c2057838100d5f/users/82b471aaac5a4793a370872a6206c917/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:01.915779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26260f2-81d0-4297-ac20-ddc37a20f955 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:01.916072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26260f2-81d0-4297-ac20-ddc37a20f955 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:01.916351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26260f2-81d0-4297-ac20-ddc37a20f955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:01.946968 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:01.963416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b26260f2-81d0-4297-ac20-ddc37a20f955 None None] MFA Rules not processed for user `82b471aaac5a4793a370872a6206c917`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:01.969368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b26260f2-81d0-4297-ac20-ddc37a20f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:02.023271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf510a-c86e-4db6-b868-1003f4bdfc63 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:02.029224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:02.030879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:02.031267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:02.031641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] PATH_INFO: `/v3/users/d1c89c198a364b05998841b87b22c76d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:02.032978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c89c198a364b05998841b87b22c76d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:02.035025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:02.037907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 980/1950] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:02.056989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26260f2-81d0-4297-ac20-ddc37a20f955 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:02.120284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26260f2-81d0-4297-ac20-ddc37a20f955 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:02.124487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a57d-316f-4573-a42e-9bf95bf17fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:02.125139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a57d-316f-4573-a42e-9bf95bf17fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:02.125422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a57d-316f-4573-a42e-9bf95bf17fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:02.125697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a57d-316f-4573-a42e-9bf95bf17fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:02.134308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a57d-316f-4573-a42e-9bf95bf17fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:02.135742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a57d-316f-4573-a42e-9bf95bf17fa5 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:02.145313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 981/1951] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:02.296103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:02.309634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c89c198a364b05998841b87b22c76d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:02.310697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c89c198a364b05998841b87b22c76d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:02.311954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c89c198a364b05998841b87b22c76d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:02.335346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c89c198a364b05998841b87b22c76d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:02.362405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:02.446839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c89c198a364b05998841b87b22c76d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:02.460412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 971/1952] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:01 2022] DELETE /identity/v3/users/d1c89c198a364b05998841b87b22c76d => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:02.464857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:02.465143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:02.465416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:02.485289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:02.486987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:02.559811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 982/1953] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:02.570442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:02.571844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:02.629639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e406c26-609b-4a70-b681-cad8b629b142 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:02.633567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:02.634325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:02.634656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:02.634935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:02.644959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:02.646487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:02.659235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 972/1954] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:03.119136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:03.120699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:03.134358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159214b-3c7b-4416-a00a-63d772ec1d8e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:03.139891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:03.141223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.141505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.141779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] PATH_INFO: `/v3/users/50b25b11d00341b5ac5d3e25263eb17f` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.143128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b25b11d00341b5ac5d3e25263eb17f)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:03.145078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:03.232432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:03.234735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:03.248807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc36873-b365-44e0-b8ce-1a3a404e2e6f service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:03.253822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:03.254723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.255024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.255289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] PATH_INFO: `/v3/projects/b79e365a60bf4723adf5ef861272ff48` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.256544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79e365a60bf4723adf5ef861272ff48)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:03.258754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:03.435444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79e365a60bf4723adf5ef861272ff48', 'request_id': 'req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.437652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b25b11d00341b5ac5d3e25263eb17f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.438487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b25b11d00341b5ac5d3e25263eb17f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.446421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b25b11d00341b5ac5d3e25263eb17f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.521888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79e365a60bf4723adf5ef861272ff48', 'request_id': 'req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.528212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79e365a60bf4723adf5ef861272ff48', 'request_id': 'req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.540125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] The token cache is being invalidate because project b79e365a60bf4723adf5ef861272ff48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:03.540419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.541068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.542648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 973/1955] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:03 2022] DELETE /identity/v3/projects/b79e365a60bf4723adf5ef861272ff48 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:03.544356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:03.546425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:03.590555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b25b11d00341b5ac5d3e25263eb17f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.603856 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8ec71a2b-7a80-40e6-95a7-45fcfaa10555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b25b11d00341b5ac5d3e25263eb17f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.609692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb1b90-57de-40fa-ba89-ed87d5fd62ec admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:03.610518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 983/1956] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:03 2022] DELETE /identity/v3/users/50b25b11d00341b5ac5d3e25263eb17f => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:03.612100 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c19444-21f4-47a8-9787-67971ae9cec8 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.612393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c19444-21f4-47a8-9787-67971ae9cec8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.612668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c19444-21f4-47a8-9787-67971ae9cec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.632217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-95c19444-21f4-47a8-9787-67971ae9cec8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:03.637503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-95c19444-21f4-47a8-9787-67971ae9cec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:03.667819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:03.668753 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.669032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.669305 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] PATH_INFO: `/v3/users/c87aa8ce243f4c1e8ed1eb48cf324cd5` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.671050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87aa8ce243f4c1e8ed1eb48cf324cd5)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:03.673001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:03.693125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 984/1957] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:03.694458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961ca33-2b32-44a2-b1a9-3bf27436500b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.694749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961ca33-2b32-44a2-b1a9-3bf27436500b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.695028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961ca33-2b32-44a2-b1a9-3bf27436500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.712343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8961ca33-2b32-44a2-b1a9-3bf27436500b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:03.713540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8961ca33-2b32-44a2-b1a9-3bf27436500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:03.721244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 985/1958] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:03.723478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.723769 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.724081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.742274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:03.743460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:03.752133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 986/1959] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:03.752954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:03.754165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:03.808947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e300df-3eee-47ee-afef-3bd4884be933 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:03.813296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:03.813951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.814263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.814545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.823904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:03.824366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:03.831548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:03.835747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 987/1960] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:03.837103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:03.850078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2628c85-caa7-4e3d-9840-9232b5cd7542 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:03.854978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:03.855771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.856073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.856352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.857085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:03.857571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:03.859105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:03.874589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 988/1961] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:03.875655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:03.886155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c115a-96f5-44b3-9547-401fa28a2893 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:03.893283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:03.894455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:03.894920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:03.895399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:03.896368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:03.900581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:03.902333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87aa8ce243f4c1e8ed1eb48cf324cd5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.903313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87aa8ce243f4c1e8ed1eb48cf324cd5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.904174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87aa8ce243f4c1e8ed1eb48cf324cd5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:03.913459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:03.918762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87aa8ce243f4c1e8ed1eb48cf324cd5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:04.053236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-91b10c74-fd10-4fb0-ba37-509870d99fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87aa8ce243f4c1e8ed1eb48cf324cd5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:04.064682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 989/1962] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:03 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:04.068930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.070580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:04.072409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 974/1963] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:03 2022] DELETE /identity/v3/users/c87aa8ce243f4c1e8ed1eb48cf324cd5 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:04.077111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.077738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.077738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.108485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:04.110403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:04.169619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e5437-ded7-49c6-bf1d-95c04bfa742c admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:04.195510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 975/1964] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:04.211219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.213862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:04.253015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:04.253820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.253968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.254894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.255004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:04.256930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:04.278196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0c9b-d3ab-4c2d-af8a-cb0070d22bd9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:04.282066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:04.282830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.283292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.283571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.293241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:04.295335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:04.305620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 976/1965] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:04.356166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.356166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:04.368661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d4aa-17b0-4a2d-9a0f-4bc580f7521b service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:04.374310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:04.375176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.375462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.375739 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] PATH_INFO: `/v3/projects/c239e6e2282943babee45377c94f848f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.377439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c239e6e2282943babee45377c94f848f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:04.379077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:04.414498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 990/1966] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:04 2022] POST /identity/v3/users => generated 289 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:04.423160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.432945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f5d07-45ef-4dcd-a0c2-32205fb2f8f0 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:04.438758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921006-8e89-4077-9928-37a50563266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:04.439507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921006-8e89-4077-9928-37a50563266b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.439860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921006-8e89-4077-9928-37a50563266b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.440275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921006-8e89-4077-9928-37a50563266b admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.441126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16921006-8e89-4077-9928-37a50563266b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:04.442840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16921006-8e89-4077-9928-37a50563266b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:04.453442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 991/1967] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:04.457584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921006-8e89-4077-9928-37a50563266b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.467555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921006-8e89-4077-9928-37a50563266b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:04.472260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:04.476744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.476744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.476744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.476744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:04.476744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:04.487028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 992/1968] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:04.491356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.493688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:04.501469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65de66-2104-40b1-8e13-aca31332406e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:04.506078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dc3e3-2593-4ac4-92eb-a296c3a5fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:04.510313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dc3e3-2593-4ac4-92eb-a296c3a5fe68 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.510313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dc3e3-2593-4ac4-92eb-a296c3a5fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.510313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dc3e3-2593-4ac4-92eb-a296c3a5fe68 admin admin] PATH_INFO: `/v3/projects/aed1573ca3ab4b8d83892f16ba1ff114/users/fba0678ca4ed484ca8c1c6bb75264076/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.523387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614dc3e3-2593-4ac4-92eb-a296c3a5fe68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed1573ca3ab4b8d83892f16ba1ff114, user_id=fba0678ca4ed484ca8c1c6bb75264076, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:04.527247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614dc3e3-2593-4ac4-92eb-a296c3a5fe68 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:04.706835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 993/1969] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:04 2022] PUT /identity/v3/projects/aed1573ca3ab4b8d83892f16ba1ff114/users/fba0678ca4ed484ca8c1c6bb75264076/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:04.720288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e19952-a4ed-4ab8-9294-408711b427a6 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.720455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c239e6e2282943babee45377c94f848f', 'request_id': 'req-e30ed5ab-342c-4fd4-be3b-d17a2f761360'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:04.720640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e19952-a4ed-4ab8-9294-408711b427a6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.720927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e19952-a4ed-4ab8-9294-408711b427a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.754530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-15e19952-a4ed-4ab8-9294-408711b427a6 None None] MFA Rules not processed for user `fba0678ca4ed484ca8c1c6bb75264076`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:04.754530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-15e19952-a4ed-4ab8-9294-408711b427a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:04.817554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 994/1970] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:04.820530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e19952-a4ed-4ab8-9294-408711b427a6 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.886018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e19952-a4ed-4ab8-9294-408711b427a6 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:04.891868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:04.892686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:04.892840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:04.893347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:04.894046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:04.896625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:04.937849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c239e6e2282943babee45377c94f848f', 'request_id': 'req-e30ed5ab-342c-4fd4-be3b-d17a2f761360'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:04.944524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c239e6e2282943babee45377c94f848f', 'request_id': 'req-e30ed5ab-342c-4fd4-be3b-d17a2f761360'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:04.957703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] The token cache is being invalidate because project c239e6e2282943babee45377c94f848f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:04.958100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:04.963120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:04.965293 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 977/1971] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:04 2022] DELETE /identity/v3/projects/c239e6e2282943babee45377c94f848f => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:04.966101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:04.967515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:05.009737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 995/1972] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:04 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:05.010715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:05.014030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:05.058132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed5ab-342c-4fd4-be3b-d17a2f761360 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:05.089344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93191bda-4f1f-4506-be0f-d52ddf78fc53 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:05.100265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:05.101189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:05.101561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:05.101895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] PATH_INFO: `/v3/users/8cf121e25557465184db7520167d476f` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:05.103138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf121e25557465184db7520167d476f)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:05.105313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:05.139161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:05.139432 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:05.139714 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:05.139988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] PATH_INFO: `/v3/projects/c90d566db2284741b688901399b49e56` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:05.141181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90d566db2284741b688901399b49e56)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:05.143194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:05.350043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90d566db2284741b688901399b49e56', 'request_id': 'req-0588389f-8dff-432b-9c67-eec898584de5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.350736 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf121e25557465184db7520167d476f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.351289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf121e25557465184db7520167d476f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.360001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf121e25557465184db7520167d476f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.418794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:05.481125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90d566db2284741b688901399b49e56', 'request_id': 'req-0588389f-8dff-432b-9c67-eec898584de5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.488302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90d566db2284741b688901399b49e56', 'request_id': 'req-0588389f-8dff-432b-9c67-eec898584de5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.500344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] The token cache is being invalidate because project c90d566db2284741b688901399b49e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:05.500634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.501300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.502964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 978/1973] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:04 2022] DELETE /identity/v3/projects/c90d566db2284741b688901399b49e56 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:05.503778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:05.505673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:05.543859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf121e25557465184db7520167d476f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.557425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf121e25557465184db7520167d476f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:05.564253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 996/1974] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:05 2022] DELETE /identity/v3/users/8cf121e25557465184db7520167d476f => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:05.568055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:05.570648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:05.576805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588389f-8dff-432b-9c67-eec898584de5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:05.641450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:05.642137 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:05.642268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243691e2-2e57-4d7a-8819-0cbb530d300e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:05.642451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:05.642731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:05.643475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:05.645466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:05.647510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:05.648864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:05.649168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:05.649445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] PATH_INFO: `/v3/users/2de9380237254ed1ac8de91f1607c398` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:05.651535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de9380237254ed1ac8de91f1607c398)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:05.656538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:05.864994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 979/1975] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:05 2022] POST /identity/v3/users => generated 295 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:05.868678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:05.879033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde8b7f-1359-479b-8645-a4b28302a0b3 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:05.883931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:05.884601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:05.884890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:05.885166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:05.885899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:05.887581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:05.898429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 980/1976] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:05.909125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:05.920802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65099f5-2823-4650-aaaa-3bfdd25e78cc admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:05.925637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:05.926571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:05.926901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:05.927177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] PATH_INFO: `/v3/projects/a863fd73477244cc893d203c6c9dfea1/users/752acb12d8fe4ac9805005a8ec53f220/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:05.942690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a863fd73477244cc893d203c6c9dfea1, user_id=752acb12d8fe4ac9805005a8ec53f220, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:05.945365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.003940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de9380237254ed1ac8de91f1607c398'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.004640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de9380237254ed1ac8de91f1607c398'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.011914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de9380237254ed1ac8de91f1607c398'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.121180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de9380237254ed1ac8de91f1607c398'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.133440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 981/1977] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:05 2022] PUT /identity/v3/projects/a863fd73477244cc893d203c6c9dfea1/users/752acb12d8fe4ac9805005a8ec53f220/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:06.134667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:06.136083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de9380237254ed1ac8de91f1607c398'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.136652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.143076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 997/1978] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:05 2022] DELETE /identity/v3/users/2de9380237254ed1ac8de91f1607c398 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:06.168965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:06.171924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.216201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b746366-8a15-4450-8612-acf9cc1fed98 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:06.267077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eaf99-94e5-4acb-9683-efec86ee7616 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:06.272694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:06.274157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.274563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.275024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] PATH_INFO: `/v3/projects/e98e5c48a2de4d8cb291c3769b02718a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.276324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98e5c48a2de4d8cb291c3769b02718a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:06.278910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.281603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:06.282354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.282694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.282979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.283715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:06.285371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.296189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 982/1979] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:06.300825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:06.310491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a88ca8-ce4d-4f55-936a-e42bb74fc9a3 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:06.315220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cac2a-65bc-4e3d-b64f-4fa7c1bc9a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:06.316097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cac2a-65bc-4e3d-b64f-4fa7c1bc9a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.316386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cac2a-65bc-4e3d-b64f-4fa7c1bc9a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.316658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cac2a-65bc-4e3d-b64f-4fa7c1bc9a10 admin admin] PATH_INFO: `/v3/projects/a863fd73477244cc893d203c6c9dfea1/users/752acb12d8fe4ac9805005a8ec53f220/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.318641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cac2a-65bc-4e3d-b64f-4fa7c1bc9a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a863fd73477244cc893d203c6c9dfea1, user_id=752acb12d8fe4ac9805005a8ec53f220, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:06.321326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cac2a-65bc-4e3d-b64f-4fa7c1bc9a10 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.331569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 983/1980] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:06 2022] PUT /identity/v3/projects/a863fd73477244cc893d203c6c9dfea1/users/752acb12d8fe4ac9805005a8ec53f220/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:06.343854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75927348-8b43-405e-9eca-7928938cdcef None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.344292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75927348-8b43-405e-9eca-7928938cdcef None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.344582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75927348-8b43-405e-9eca-7928938cdcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.374754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-75927348-8b43-405e-9eca-7928938cdcef None None] MFA Rules not processed for user `752acb12d8fe4ac9805005a8ec53f220`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:06.376705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-75927348-8b43-405e-9eca-7928938cdcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.393571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98e5c48a2de4d8cb291c3769b02718a', 'request_id': 'req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.400157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98e5c48a2de4d8cb291c3769b02718a', 'request_id': 'req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.452344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 984/1981] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:06.462980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75927348-8b43-405e-9eca-7928938cdcef None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:06.464469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-75927348-8b43-405e-9eca-7928938cdcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.523539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75927348-8b43-405e-9eca-7928938cdcef None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:06.527395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:06.528055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.528493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.528789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.530930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.543466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98e5c48a2de4d8cb291c3769b02718a', 'request_id': 'req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.554947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] The token cache is being invalidate because project e98e5c48a2de4d8cb291c3769b02718a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:06.560545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.560545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2c3df0d9-16e3-48c4-94d9-4956d25a7e9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.560545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 998/1982] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:06 2022] DELETE /identity/v3/projects/e98e5c48a2de4d8cb291c3769b02718a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:06.562666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017ca5-6ac4-4bb7-9772-95de361facfd None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.562960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017ca5-6ac4-4bb7-9772-95de361facfd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.563240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017ca5-6ac4-4bb7-9772-95de361facfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.594829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-01017ca5-6ac4-4bb7-9772-95de361facfd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:06.596368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:06.596616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-01017ca5-6ac4-4bb7-9772-95de361facfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.597865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.607088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 985/1983] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:06.660553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 999/1984] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:06.667435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:06.668452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4efd01-91df-49aa-834e-447aee91ab2d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.669664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.669787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4efd01-91df-49aa-834e-447aee91ab2d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.669787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4efd01-91df-49aa-834e-447aee91ab2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.683447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00145847-13f8-4fcf-9ba3-5ebc9cc301e0 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:06.690007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:06.691073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.691431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.691852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] PATH_INFO: `/v3/projects/711d47f41aaf405b80d9d06c856a0e49` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.693212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711d47f41aaf405b80d9d06c856a0e49)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:06.695299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.707019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8f4efd01-91df-49aa-834e-447aee91ab2d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:06.718107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8f4efd01-91df-49aa-834e-447aee91ab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:06.722098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1000/1985] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:06.723271 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4efd01-91df-49aa-834e-447aee91ab2d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:06.734778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4efd01-91df-49aa-834e-447aee91ab2d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:06.740963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:06.741646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.741985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.742572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.743434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:06.743753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:06.746377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.760809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1001/1986] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:06.765165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:06.776583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb364ce2-e208-423f-854c-3c6597bdd27d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:06.782133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:06.782828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:06.783106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:06.783380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:06.784067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:06.785970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:06.905857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711d47f41aaf405b80d9d06c856a0e49', 'request_id': 'req-78a8a535-211d-4f2f-8836-d8bb118424f3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:06.928833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1002/1987] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:06 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:06.933283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.010623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c566da-c54e-4cca-afb4-dcbd3a1b53f7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.015416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.016276 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.016568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.016989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.017957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.019333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711d47f41aaf405b80d9d06c856a0e49', 'request_id': 'req-78a8a535-211d-4f2f-8836-d8bb118424f3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:07.019875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.026268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711d47f41aaf405b80d9d06c856a0e49', 'request_id': 'req-78a8a535-211d-4f2f-8836-d8bb118424f3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:07.039582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] The token cache is being invalidate because project 711d47f41aaf405b80d9d06c856a0e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:07.039874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:07.040545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-78a8a535-211d-4f2f-8836-d8bb118424f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:07.042407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 986/1988] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:06 2022] DELETE /identity/v3/projects/711d47f41aaf405b80d9d06c856a0e49 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:07.047729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdca89-eb1f-49ad-80a3-2fccbfbbf824 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.048069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdca89-eb1f-49ad-80a3-2fccbfbbf824 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.048299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdca89-eb1f-49ad-80a3-2fccbfbbf824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.069076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-38bdca89-eb1f-49ad-80a3-2fccbfbbf824 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:07.070321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-38bdca89-eb1f-49ad-80a3-2fccbfbbf824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:07.078851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 987/1989] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:07.086543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af7873-87a0-463c-98c1-a158cb688c35 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.086831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af7873-87a0-463c-98c1-a158cb688c35 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.088894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af7873-87a0-463c-98c1-a158cb688c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.104941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-28af7873-87a0-463c-98c1-a158cb688c35 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:07.106221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-28af7873-87a0-463c-98c1-a158cb688c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:07.115185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 988/1990] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:07.117675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1003/1991] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:06 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:07.120533 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af7873-87a0-463c-98c1-a158cb688c35 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.122873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.124867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:07.131067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af7873-87a0-463c-98c1-a158cb688c35 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.136585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.137263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.138695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.138855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.139577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:07.139881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.140919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eff63-2574-47a1-8377-f0aa6c1194b9 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.141634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.146759 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.147590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.147930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.148269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.149136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.150856 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.157271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 989/1992] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:57:07.158309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.165072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1004/1993] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:07.166157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.168549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f830bb7-1544-40f8-a3ce-89f458609001 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.173960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.174657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.174942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.180098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.180098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.180279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93367304-d8c9-405a-8a28-1fe248748e9e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.180371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.182558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.183212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.183493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.183766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.185691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.187351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.197708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1005/1994] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:07.201287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:07.201890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.211431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f79e2e-97a8-4b64-8a18-e1a49e659533 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.216198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1d435-f589-4a6a-b479-cd55cadae646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.217052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1d435-f589-4a6a-b479-cd55cadae646 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.217329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1d435-f589-4a6a-b479-cd55cadae646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.217601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1d435-f589-4a6a-b479-cd55cadae646 admin admin] PATH_INFO: `/v3/projects/e8cd1c59edd348d8b12fc3cf4aed9e05/users/826a18d3adfc414197d32f597166db88/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.232245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1d435-f589-4a6a-b479-cd55cadae646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8cd1c59edd348d8b12fc3cf4aed9e05, user_id=826a18d3adfc414197d32f597166db88, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.234997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1d435-f589-4a6a-b479-cd55cadae646 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.286418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:07.293481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 990/1995] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:07 2022] POST /identity/v3/projects => generated 361 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:07.301697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.357385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfe9af-dc48-414e-9e1a-cc7f54dbb1bd admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.361438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1006/1996] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:07 2022] PUT /identity/v3/projects/e8cd1c59edd348d8b12fc3cf4aed9e05/users/826a18d3adfc414197d32f597166db88/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:07.371272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.371562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.372598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.414187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] MFA Rules not processed for user `826a18d3adfc414197d32f597166db88`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:07.415748 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:07.418883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.425209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.425209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.425209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.425209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.425209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.482677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1007/1997] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:07.494841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.504641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:07.554580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15cd2-c99a-439a-b26b-6dc70737e6cb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.558251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.558906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.559232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.559551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.569043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.571947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.578299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 991/1998] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:07 2022] POST /identity/v3/users => generated 281 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:07.582011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1008/1999] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:07.582261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.594730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50769c01-3002-4944-b75a-11123c61387b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.603981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.604916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.605604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.606706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.606820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.607531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.609163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.615942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54600e6-a289-437a-b164-41d566b00ccc service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.622713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 992/2000] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:07.622847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.622847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.622847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.622847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] PATH_INFO: `/v3/users/826a18d3adfc414197d32f597166db88` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.623954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.624079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=826a18d3adfc414197d32f597166db88)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.625891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.633725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4dcd8-a18b-49ce-9ece-72346281aac7 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.638595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.639262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.639550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.639832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.640560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.642165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.652750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 993/2001] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:07.661895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:07.672233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb8916-8ba3-4ab2-818b-b3defcbe3851 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:07.676817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3dff2-bced-4019-890a-6b4091073d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:07.677680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3dff2-bced-4019-890a-6b4091073d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.677972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3dff2-bced-4019-890a-6b4091073d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.678389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3dff2-bced-4019-890a-6b4091073d6d admin admin] PATH_INFO: `/v3/projects/9b13fbf6ebb04b47b41b31017da8b026/users/a3a3ae5069b441f7b96edb0c9d3ff849/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.692773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3dff2-bced-4019-890a-6b4091073d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b13fbf6ebb04b47b41b31017da8b026, user_id=a3a3ae5069b441f7b96edb0c9d3ff849, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:07.695499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3dff2-bced-4019-890a-6b4091073d6d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:07.846557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 994/2002] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:07 2022] PUT /identity/v3/projects/9b13fbf6ebb04b47b41b31017da8b026/users/a3a3ae5069b441f7b96edb0c9d3ff849/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:07.867288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:07.867581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:07.867862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:07.895315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] MFA Rules not processed for user `a3a3ae5069b441f7b96edb0c9d3ff849`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:07.896571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:07.960377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 995/2003] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:07.984782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826a18d3adfc414197d32f597166db88'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:07.985806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826a18d3adfc414197d32f597166db88'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:07.994021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826a18d3adfc414197d32f597166db88'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.004634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:08.006444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:08.069697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661970-c0c4-457f-ae2d-6de3c5c92cd7 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:08.073582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:08.074298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:08.075289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:08.075593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:08.077899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:08.089608 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:08.091924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:08.102962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826a18d3adfc414197d32f597166db88'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.103486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 996/2004] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:08.120075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bd50c461-2023-48f4-88fa-8cbe0e5d70bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826a18d3adfc414197d32f597166db88'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.123589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:08.125609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:08.130904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1009/2005] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:07 2022] DELETE /identity/v3/users/826a18d3adfc414197d32f597166db88 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:08.136229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:08.136715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:08.137770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:08.162691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:08.164023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:08.215723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b475f9-adb8-4cff-ba94-f8136f2ea2dd service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:08.245356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1010/2006] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:08.255338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:08.257075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:08.285101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:08.286056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:08.286527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:08.286894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] PATH_INFO: `/v3/users/a3a3ae5069b441f7b96edb0c9d3ff849` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:08.288304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a3ae5069b441f7b96edb0c9d3ff849)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:08.290888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:08.319878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f46e4-4f49-436d-9d81-cde8ed418b15 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:08.323988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ae684-1e81-421e-8420-6ffe2e5a9f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:08.325200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ae684-1e81-421e-8420-6ffe2e5a9f5a service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:08.325516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ae684-1e81-421e-8420-6ffe2e5a9f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:08.325804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ae684-1e81-421e-8420-6ffe2e5a9f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:08.335481 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ae684-1e81-421e-8420-6ffe2e5a9f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:08.337183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ae684-1e81-421e-8420-6ffe2e5a9f5a service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:08.343647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:08.348516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1011/2007] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:08.583257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a3ae5069b441f7b96edb0c9d3ff849'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.583912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a3ae5069b441f7b96edb0c9d3ff849'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.584530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a3ae5069b441f7b96edb0c9d3ff849'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.598896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a3ae5069b441f7b96edb0c9d3ff849'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.708482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a3ae5069b441f7b96edb0c9d3ff849'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:08.722072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 997/2008] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:08 2022] DELETE /identity/v3/users/a3a3ae5069b441f7b96edb0c9d3ff849 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:08.726669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:08.726960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:08.727234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:08.747575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:08.748830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:08.818162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1012/2009] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:08.827354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:08.828992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:08.887681 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb101fe3-fd31-463a-be49-9f85238ca5ba admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:08.891580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:08.892234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:08.892512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:08.892815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:08.902008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:08.903755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:08.913580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 998/2010] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:08.962932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:08.967337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:09.010623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18795d5e-572e-4166-a50b-02afd4dd619b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:09.015934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:09.016814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:09.017093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:09.017366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] PATH_INFO: `/v3/projects/e8cd1c59edd348d8b12fc3cf4aed9e05` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:09.018663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8cd1c59edd348d8b12fc3cf4aed9e05)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:09.028470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:09.188534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8cd1c59edd348d8b12fc3cf4aed9e05', 'request_id': 'req-53753180-dc61-47b3-bf15-bf9816024425'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.194896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8cd1c59edd348d8b12fc3cf4aed9e05', 'request_id': 'req-53753180-dc61-47b3-bf15-bf9816024425'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.297224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8cd1c59edd348d8b12fc3cf4aed9e05', 'request_id': 'req-53753180-dc61-47b3-bf15-bf9816024425'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.313750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] The token cache is being invalidate because project e8cd1c59edd348d8b12fc3cf4aed9e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:09.314043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.314865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-53753180-dc61-47b3-bf15-bf9816024425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.316477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1013/2011] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:08 2022] DELETE /identity/v3/projects/e8cd1c59edd348d8b12fc3cf4aed9e05 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:09.555896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:09.557336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:09.580792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827a4e-b7e8-4d14-b45a-2f4649e13721 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:09.649929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:09.650852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:09.651139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:09.651540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] PATH_INFO: `/v3/projects/9b13fbf6ebb04b47b41b31017da8b026` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:09.653033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b13fbf6ebb04b47b41b31017da8b026)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:09.654859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:09.806614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b13fbf6ebb04b47b41b31017da8b026', 'request_id': 'req-e36ad7d2-755a-403a-85de-b17bd3ff7b31'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.925198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b13fbf6ebb04b47b41b31017da8b026', 'request_id': 'req-e36ad7d2-755a-403a-85de-b17bd3ff7b31'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.932031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b13fbf6ebb04b47b41b31017da8b026', 'request_id': 'req-e36ad7d2-755a-403a-85de-b17bd3ff7b31'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.944094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] The token cache is being invalidate because project 9b13fbf6ebb04b47b41b31017da8b026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:09.944386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.945072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e36ad7d2-755a-403a-85de-b17bd3ff7b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:09.946992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 999/2012] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:09 2022] DELETE /identity/v3/projects/9b13fbf6ebb04b47b41b31017da8b026 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:09.953407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c172c-c07f-4baa-bf1d-4a55641f20a3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:09.953702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c172c-c07f-4baa-bf1d-4a55641f20a3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:09.953987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c172c-c07f-4baa-bf1d-4a55641f20a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:09.972283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-356c172c-c07f-4baa-bf1d-4a55641f20a3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:09.973501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-356c172c-c07f-4baa-bf1d-4a55641f20a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:10.036529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1014/2013] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:10.044801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df10a7-1a87-446f-8ee1-ce91c3277114 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.048240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df10a7-1a87-446f-8ee1-ce91c3277114 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.048240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df10a7-1a87-446f-8ee1-ce91c3277114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.065588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f6df10a7-1a87-446f-8ee1-ce91c3277114 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:10.066947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f6df10a7-1a87-446f-8ee1-ce91c3277114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:10.076504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1000/2014] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:10.081278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c172c-c07f-4baa-bf1d-4a55641f20a3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:10.091294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c172c-c07f-4baa-bf1d-4a55641f20a3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:10.096002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:10.096718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.097016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.097289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.098024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:10.098379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:10.099990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:10.112304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1015/2015] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:57:10.116708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df10a7-1a87-446f-8ee1-ce91c3277114 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:10.126941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df10a7-1a87-446f-8ee1-ce91c3277114 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:10.133015 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:10.133684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.133998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.134315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.135014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:10.137594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:10.275445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1001/2016] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:10 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:10.279691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:10.289630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d44545-038e-4ef0-9052-a70931e3eb5d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:10.345163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:10.345845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.346107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.346420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.347109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:10.349019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:10.486585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1016/2017] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:10 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:10.490618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:10.501873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4f41-5436-4534-a4df-dd00054326e5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:10.507622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:10.508454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.508839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.509215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.510051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:10.511902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:10.515971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:10.524350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1002/2018] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:10.529889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:10.541138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad826303-de91-4209-a28b-cda478151c99 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:10.547128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e1573-8051-4017-9db8-05269e16ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:10.552884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e1573-8051-4017-9db8-05269e16ace9 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.552884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e1573-8051-4017-9db8-05269e16ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.552884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e1573-8051-4017-9db8-05269e16ace9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.552884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e1573-8051-4017-9db8-05269e16ace9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:10.552884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e1573-8051-4017-9db8-05269e16ace9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:10.563661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1017/2019] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:10.574821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:10.588355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d33d8a-f337-489a-b63c-d558e86ae2da admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:10.591885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:10.593019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.593367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.593709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] PATH_INFO: `/v3/projects/aad34ced0e7f465f843b4393dd09fbcf/users/9a4bf2b0654c4fba95cd42488791996a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.609045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad34ced0e7f465f843b4393dd09fbcf, user_id=9a4bf2b0654c4fba95cd42488791996a, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:10.612325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:10.737180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1003/2020] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:10 2022] PUT /identity/v3/projects/aad34ced0e7f465f843b4393dd09fbcf/users/9a4bf2b0654c4fba95cd42488791996a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:10.748659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2078e-4bd3-4e24-b814-c851a63f8262 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.757922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2078e-4bd3-4e24-b814-c851a63f8262 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.757922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2078e-4bd3-4e24-b814-c851a63f8262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.775964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-10e2078e-4bd3-4e24-b814-c851a63f8262 None None] MFA Rules not processed for user `9a4bf2b0654c4fba95cd42488791996a`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:10.777766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-10e2078e-4bd3-4e24-b814-c851a63f8262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:10.842337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1018/2021] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:10.853262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:10.855072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:10.912244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391979e4-4367-4a32-8102-e8b3378fcd57 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:10.915810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:10.916453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:10.916733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:10.917007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:10.926101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:10.927565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:10.936586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1004/2022] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:11.068238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:12.850595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:13.297152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2078e-4bd3-4e24-b814-c851a63f8262 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:13.306875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2078e-4bd3-4e24-b814-c851a63f8262 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:13.311185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907854e4-d58a-4584-bbe7-5118073c5d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:13.311829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907854e4-d58a-4584-bbe7-5118073c5d52 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:13.312111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907854e4-d58a-4584-bbe7-5118073c5d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:13.312388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907854e4-d58a-4584-bbe7-5118073c5d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:13.314137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-907854e4-d58a-4584-bbe7-5118073c5d52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:13.378225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907854e4-d58a-4584-bbe7-5118073c5d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:13.380025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907854e4-d58a-4584-bbe7-5118073c5d52 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:13.390634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1019/2023] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:13.741433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:15.060931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:16.225968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:17.077686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:17.111936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:17.814222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:18.048846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:18.644362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:18.709467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf07b6-829f-4d0b-8d86-5cf87cd5d466 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:18.715251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:18.716125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:18.716409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:18.716680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] PATH_INFO: `/v3/users/9a4bf2b0654c4fba95cd42488791996a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:18.717865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4bf2b0654c4fba95cd42488791996a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:18.719839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:19.061859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4bf2b0654c4fba95cd42488791996a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.062532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4bf2b0654c4fba95cd42488791996a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.063143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4bf2b0654c4fba95cd42488791996a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.076052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4bf2b0654c4fba95cd42488791996a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.207654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4bf2b0654c4fba95cd42488791996a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.220809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1005/2024] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:18 2022] DELETE /identity/v3/users/9a4bf2b0654c4fba95cd42488791996a => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:19.225427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:19.225918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:19.226015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:19.248751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:19.249963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:19.308015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1020/2025] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:19.317818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:19.319221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:19.378372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a58d24-22aa-4090-ac85-7209c2760b1c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:19.381962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:19.382755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:19.383039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:19.383314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:19.393058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:19.394544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:19.404181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1006/2026] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:19.592647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:19.594036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:19.606334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313a627-2876-42e5-ac2e-510720ac852d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:19.611009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:19.615616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:19.615616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:19.615616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] PATH_INFO: `/v3/users/82b471aaac5a4793a370872a6206c917` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:19.615616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b471aaac5a4793a370872a6206c917)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:19.615616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:19.801853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:19.895834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b471aaac5a4793a370872a6206c917'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.896556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b471aaac5a4793a370872a6206c917'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.904329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b471aaac5a4793a370872a6206c917'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.926934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:19.928938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:19.944695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82842b3-283b-4ea2-b09c-ba39cb159bbf service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:19.953774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:19.953774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:19.953774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:19.953774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] PATH_INFO: `/v3/projects/aad34ced0e7f465f843b4393dd09fbcf` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:19.954976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad34ced0e7f465f843b4393dd09fbcf)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:19.957173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:19.973503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b471aaac5a4793a370872a6206c917'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.988520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e0972dda-1798-4c6b-8f4a-6468ceea311b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b471aaac5a4793a370872a6206c917'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:19.995848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1021/2027] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:19 2022] DELETE /identity/v3/users/82b471aaac5a4793a370872a6206c917 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:19.999946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.000502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.000778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.019541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:20.020828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:20.089319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1022/2028] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:20.098947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.100372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:20.107551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad34ced0e7f465f843b4393dd09fbcf', 'request_id': 'req-588b2075-7934-4820-94ef-fbdeedd30cf2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:20.156187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46368631-e467-40c7-aee9-d8632d4317a5 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.204990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02977651-bd03-4d1e-95f8-c27f22c6a837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.205648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02977651-bd03-4d1e-95f8-c27f22c6a837 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.205929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02977651-bd03-4d1e-95f8-c27f22c6a837 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.206234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02977651-bd03-4d1e-95f8-c27f22c6a837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.207047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:20.266265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02977651-bd03-4d1e-95f8-c27f22c6a837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.267718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02977651-bd03-4d1e-95f8-c27f22c6a837 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:20.278612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1023/2029] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:20.292486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad34ced0e7f465f843b4393dd09fbcf', 'request_id': 'req-588b2075-7934-4820-94ef-fbdeedd30cf2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:20.298476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad34ced0e7f465f843b4393dd09fbcf', 'request_id': 'req-588b2075-7934-4820-94ef-fbdeedd30cf2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:20.309888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] The token cache is being invalidate because project aad34ced0e7f465f843b4393dd09fbcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:20.310207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:20.310868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-588b2075-7934-4820-94ef-fbdeedd30cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:20.314270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1007/2030] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:19 2022] DELETE /identity/v3/projects/aad34ced0e7f465f843b4393dd09fbcf => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:20.317404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e56a-2fde-4182-bd8f-231a694c4df8 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.317768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e56a-2fde-4182-bd8f-231a694c4df8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.318049 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12e56a-2fde-4182-bd8f-231a694c4df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.340147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3d12e56a-2fde-4182-bd8f-231a694c4df8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:20.341139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3d12e56a-2fde-4182-bd8f-231a694c4df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:20.349714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1024/2031] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:20.357973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4831-98c0-410a-a666-528b6399217f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.358313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4831-98c0-410a-a666-528b6399217f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.358589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4831-98c0-410a-a666-528b6399217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.376384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-12db4831-98c0-410a-a666-528b6399217f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:20.377897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-12db4831-98c0-410a-a666-528b6399217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:20.386349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1008/2032] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:20.390563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12e56a-2fde-4182-bd8f-231a694c4df8 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.401144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12e56a-2fde-4182-bd8f-231a694c4df8 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.405719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.406832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.406832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.407020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.407635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:20.407937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.409523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:20.421754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1025/2033] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:20.425967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4831-98c0-410a-a666-528b6399217f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.435288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4831-98c0-410a-a666-528b6399217f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.439960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.441278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.441278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.441278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.441850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.443791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:20.581621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1009/2034] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:20 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:20.585374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.596704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62088a-a92d-4381-9687-31f564fe7030 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.656032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.656706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.656990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.657269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.657971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.659939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:20.793486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.795548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:20.797046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1026/2035] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:20 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:20.800847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.813084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41c19c-55ee-42d9-bf2b-aee5f5b3353a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.813894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45695c5a-6fc8-4197-9d4a-1afbbb9a9aec admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.818205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.818580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.819107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.819439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.819541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.819721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.820642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] PATH_INFO: `/v3/projects/d8c2bc0c830b48b784c2057838100d5f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.820775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.821076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.821948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c2bc0c830b48b784c2057838100d5f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.822958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:20.823946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:20.837703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1027/2036] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:20.840148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.849693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3640ab-7265-4b89-a33a-2c358aed6bbe admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.855646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.856337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.856616 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.856886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.857606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.859289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:20.871672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1028/2037] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:20.875777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:20.885790 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede7f36-e432-41e5-9c2f-594db6ed266d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:20.890361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a304312-d883-411e-b6c1-7b5c4bd29e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:20.891219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a304312-d883-411e-b6c1-7b5c4bd29e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:20.891494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a304312-d883-411e-b6c1-7b5c4bd29e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:20.891765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a304312-d883-411e-b6c1-7b5c4bd29e92 admin admin] PATH_INFO: `/v3/projects/172a98bc421a4ed386942e6eb19120c4/users/66877cbe4a954ab8a1607a1a4419ab0b/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:20.907123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a304312-d883-411e-b6c1-7b5c4bd29e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172a98bc421a4ed386942e6eb19120c4, user_id=66877cbe4a954ab8a1607a1a4419ab0b, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:20.909932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a304312-d883-411e-b6c1-7b5c4bd29e92 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.001780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1029/2038] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:20 2022] PUT /identity/v3/projects/172a98bc421a4ed386942e6eb19120c4/users/66877cbe4a954ab8a1607a1a4419ab0b/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:21.010298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c2bc0c830b48b784c2057838100d5f', 'request_id': 'req-53a673b2-a7cb-4d93-bea5-19758e5eb099'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.012833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.013128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.013402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.040005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] MFA Rules not processed for user `66877cbe4a954ab8a1607a1a4419ab0b`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:21.041284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.107445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1030/2039] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:21.120114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.121517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.122394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c2bc0c830b48b784c2057838100d5f', 'request_id': 'req-53a673b2-a7cb-4d93-bea5-19758e5eb099'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.136040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c2bc0c830b48b784c2057838100d5f', 'request_id': 'req-53a673b2-a7cb-4d93-bea5-19758e5eb099'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.149475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] The token cache is being invalidate because project d8c2bc0c830b48b784c2057838100d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:21.149769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.154621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53a673b2-a7cb-4d93-bea5-19758e5eb099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.157548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1010/2040] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:20 2022] DELETE /identity/v3/projects/d8c2bc0c830b48b784c2057838100d5f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:21.162014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e937f-e473-4ed4-bad7-2e950227463f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.162395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e937f-e473-4ed4-bad7-2e950227463f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.162768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e937f-e473-4ed4-bad7-2e950227463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.182121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-152e937f-e473-4ed4-bad7-2e950227463f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:21.183460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-152e937f-e473-4ed4-bad7-2e950227463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.193614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceedf56-7b7f-4a7b-ba86-cca70a70f41c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.198075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.198768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.199052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.199333 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.201692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.214353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.215806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.225653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1031/2041] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:21.248371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1011/2042] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:21.249861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:21.255067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.255698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4023b5-d56d-4733-a517-e8c8a8dd2848 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.255944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4023b5-d56d-4733-a517-e8c8a8dd2848 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.256275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4023b5-d56d-4733-a517-e8c8a8dd2848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.256817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.274458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73e802-8457-4f33-987c-aae5dd219179 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.279671 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.280766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.281053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.281341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] PATH_INFO: `/v3/users/66877cbe4a954ab8a1607a1a4419ab0b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.282447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1d4023b5-d56d-4733-a517-e8c8a8dd2848 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:21.282575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66877cbe4a954ab8a1607a1a4419ab0b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.283656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1d4023b5-d56d-4733-a517-e8c8a8dd2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.284530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.292326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1012/2043] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:21.296433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4023b5-d56d-4733-a517-e8c8a8dd2848 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.306513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4023b5-d56d-4733-a517-e8c8a8dd2848 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.311325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.311981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.312267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.312544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.313268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:21.313578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.315267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.316230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:21.327463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1013/2044] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 14:57:21.331632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.341152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a18a3e-8188-4413-9daa-ab34f13b9a74 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.345788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.346463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.346752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.347032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.347707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.349637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.512000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1014/2045] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:21 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:21.515993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.577654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab6af4-11e4-41b6-9b52-595760ddbaae admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.582549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.583222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.583512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.583789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.584487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.586490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.611704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66877cbe4a954ab8a1607a1a4419ab0b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.616627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66877cbe4a954ab8a1607a1a4419ab0b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.619324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66877cbe4a954ab8a1607a1a4419ab0b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.710205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1015/2046] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:21 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:21.714988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.716669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.728776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b202eeb-4024-43f8-a739-7ac11eaf7aee admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.733458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.734102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.734420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.734699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.735425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.737142 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.747355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1016/2047] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:21.751079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.771195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10860-0935-4092-8c55-381abe50b6bb admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.771195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.771195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.771195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.771626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.771626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.771626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.779991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1017/2048] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:21.783804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.793135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f02927-2a1c-4a52-9ba8-d0c9635a57af admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:21.797703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02764717-4c70-407a-b38b-ea0a4163dfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:21.798604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02764717-4c70-407a-b38b-ea0a4163dfb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.798887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02764717-4c70-407a-b38b-ea0a4163dfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.799175 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02764717-4c70-407a-b38b-ea0a4163dfb8 admin admin] PATH_INFO: `/v3/projects/22a6143a5b05471e8aed517230f734e7/users/8a74a23b13e24616921c55bc6226216f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.813714 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02764717-4c70-407a-b38b-ea0a4163dfb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a6143a5b05471e8aed517230f734e7, user_id=8a74a23b13e24616921c55bc6226216f, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:21.816385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02764717-4c70-407a-b38b-ea0a4163dfb8 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:21.818900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66877cbe4a954ab8a1607a1a4419ab0b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.831778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8501825f-d0df-4edd-90e3-326df77a8865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66877cbe4a954ab8a1607a1a4419ab0b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:21.837907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1032/2049] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:21 2022] DELETE /identity/v3/users/66877cbe4a954ab8a1607a1a4419ab0b => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:21.842300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e866812-8878-486c-9998-75d6bb841197 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.842623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e866812-8878-486c-9998-75d6bb841197 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.842912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e866812-8878-486c-9998-75d6bb841197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.861321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-9e866812-8878-486c-9998-75d6bb841197 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:21.862574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9e866812-8878-486c-9998-75d6bb841197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.919377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1033/2050] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:21.929626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e866812-8878-486c-9998-75d6bb841197 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:21.931598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9e866812-8878-486c-9998-75d6bb841197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:21.968672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1018/2051] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:21 2022] PUT /identity/v3/projects/22a6143a5b05471e8aed517230f734e7/users/8a74a23b13e24616921c55bc6226216f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:21.979614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:21.982469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:21.982469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:21.986849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e866812-8878-486c-9998-75d6bb841197 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:22.006957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] MFA Rules not processed for user `8a74a23b13e24616921c55bc6226216f`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:22.009068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:22.035752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:22.036323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:22.036607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:22.036886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:22.075436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1019/2052] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:22.086386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:22.087751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:22.098201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:22.102894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:22.109233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1034/2053] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:22.144207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117cd6d-8e57-46fc-87db-676655271b8f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:22.147876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4413ad-c1fe-4cd1-a463-132b7e3ad25b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:22.148541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4413ad-c1fe-4cd1-a463-132b7e3ad25b service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:22.148850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4413ad-c1fe-4cd1-a463-132b7e3ad25b service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:22.149124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4413ad-c1fe-4cd1-a463-132b7e3ad25b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:22.157514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4413ad-c1fe-4cd1-a463-132b7e3ad25b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:22.158993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4413ad-c1fe-4cd1-a463-132b7e3ad25b service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:22.171661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1020/2054] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:57:22.441526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:22.571708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:22.573236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:22.587919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ad82a-4746-493e-9a7a-11ba6409c3ee service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:22.592741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:22.593632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:22.594063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:22.594392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] PATH_INFO: `/v3/projects/172a98bc421a4ed386942e6eb19120c4` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:22.595761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=172a98bc421a4ed386942e6eb19120c4)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:22.597754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:22.751097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172a98bc421a4ed386942e6eb19120c4', 'request_id': 'req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:22.758299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172a98bc421a4ed386942e6eb19120c4', 'request_id': 'req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:22.841249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:22.932774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172a98bc421a4ed386942e6eb19120c4', 'request_id': 'req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:22.944029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] The token cache is being invalidate because project 172a98bc421a4ed386942e6eb19120c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:22.944435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:22.945462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0d102dd4-1ce1-4028-b978-f8c9dd7f5e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:22.947347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1035/2055] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:22 2022] DELETE /identity/v3/projects/172a98bc421a4ed386942e6eb19120c4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:22.952234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35fb9-416a-4a22-a30d-2d205853de87 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:22.952523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35fb9-416a-4a22-a30d-2d205853de87 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:22.952797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35fb9-416a-4a22-a30d-2d205853de87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:22.971684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-aae35fb9-416a-4a22-a30d-2d205853de87 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:22.972330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-aae35fb9-416a-4a22-a30d-2d205853de87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:23.042937 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1021/2056] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:23.051585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3700-b713-46c9-acc0-d349b4b772db None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.051889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3700-b713-46c9-acc0-d349b4b772db None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.052210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3700-b713-46c9-acc0-d349b4b772db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.071431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-234d3700-b713-46c9-acc0-d349b4b772db None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:23.072642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-234d3700-b713-46c9-acc0-d349b4b772db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:23.084147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1036/2057] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:23.087027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35fb9-416a-4a22-a30d-2d205853de87 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:23.096661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35fb9-416a-4a22-a30d-2d205853de87 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:23.102624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:23.103306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.103738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.104020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.104752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:23.105088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:23.106773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:23.120098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1022/2058] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:23.124555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3700-b713-46c9-acc0-d349b4b772db None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:23.135175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3700-b713-46c9-acc0-d349b4b772db None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:23.140652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:23.141318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.141617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.141881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.142622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:23.150102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:23.282976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1037/2059] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:23 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:23.287696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:23.299840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869bb80-a669-4bff-bd83-94308e900b14 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:23.357181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:23.357750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.358030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.358332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.359017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:23.360947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:23.481396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1023/2060] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:23 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:23.485620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:23.497092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c1ec7-0573-475e-909f-c30a0f7cc68b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:23.503007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:23.503688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.503978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.504259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.505032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:23.506762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:23.518037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1038/2061] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:23.524986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:23.535349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c4abc-75bd-478f-8f57-8f9fe71ca659 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:23.541494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c40f2f-9b12-41cb-8d1b-7554c1e1bc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:23.542277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c40f2f-9b12-41cb-8d1b-7554c1e1bc75 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.550332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c40f2f-9b12-41cb-8d1b-7554c1e1bc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.550332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c40f2f-9b12-41cb-8d1b-7554c1e1bc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.550332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c40f2f-9b12-41cb-8d1b-7554c1e1bc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:23.550332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c40f2f-9b12-41cb-8d1b-7554c1e1bc75 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:23.559970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1024/2062] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:23.563240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:23.573642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22156f9-f364-4475-a911-794802e7ffe0 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:23.584630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:23.584630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.584630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.584630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] PATH_INFO: `/v3/projects/421ba5fd9c9d4ea0b3b3dd89069aabb2/users/4e9c4ff3d13d4332837f519214b9df37/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.601440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421ba5fd9c9d4ea0b3b3dd89069aabb2, user_id=4e9c4ff3d13d4332837f519214b9df37, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:23.604565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:23.749696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1039/2063] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:23 2022] PUT /identity/v3/projects/421ba5fd9c9d4ea0b3b3dd89069aabb2/users/4e9c4ff3d13d4332837f519214b9df37/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:23.755055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43b5a7-1671-4f39-8c1e-8239b0316bae None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.755347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43b5a7-1671-4f39-8c1e-8239b0316bae None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.755626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43b5a7-1671-4f39-8c1e-8239b0316bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.773104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-dc43b5a7-1671-4f39-8c1e-8239b0316bae None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:23.774336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dc43b5a7-1671-4f39-8c1e-8239b0316bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:23.832414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1025/2064] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:23.842283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:23.844033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:23.902072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1639b-27c3-4887-8960-73ee93ae9844 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:23.905725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8792683-c7c5-423e-89f6-94fe603a06a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:23.906431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8792683-c7c5-423e-89f6-94fe603a06a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:23.906713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8792683-c7c5-423e-89f6-94fe603a06a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:23.906990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8792683-c7c5-423e-89f6-94fe603a06a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:23.916793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8792683-c7c5-423e-89f6-94fe603a06a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:23.918263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8792683-c7c5-423e-89f6-94fe603a06a2 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:23.927583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1040/2065] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:24.035193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:24.525695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:25.139782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:25.140071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:25.140346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:25.157706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] MFA Rules not processed for user `8a74a23b13e24616921c55bc6226216f`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:25.158931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:25.218414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1026/2066] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:25.400490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:25.498132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553bb03-af02-4823-b3e8-092a02a8d5b9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:25.498451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553bb03-af02-4823-b3e8-092a02a8d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:25.498962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553bb03-af02-4823-b3e8-092a02a8d5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:25.536455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4553bb03-af02-4823-b3e8-092a02a8d5b9 None None] MFA Rules not processed for user `4e9c4ff3d13d4332837f519214b9df37`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:25.536455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4553bb03-af02-4823-b3e8-092a02a8d5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:25.607740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1041/2067] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:25.620003 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:25.621691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:25.702100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0493b-9e31-4c03-88e7-aedc016ae3da None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:25.705975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:25.706696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:25.706975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:25.707253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:25.716255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:25.717691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:25.727524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1027/2068] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:26.333568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:26.984007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:27.859935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c3ee3-3e3b-4b6f-bf44-817eedcfcc6a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:27.860471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c3ee3-3e3b-4b6f-bf44-817eedcfcc6a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:27.860471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c3ee3-3e3b-4b6f-bf44-817eedcfcc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:27.879892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c28c3ee3-3e3b-4b6f-bf44-817eedcfcc6a None None] MFA Rules not processed for user `4e9c4ff3d13d4332837f519214b9df37`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:27.882152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c28c3ee3-3e3b-4b6f-bf44-817eedcfcc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:27.889032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1042/2069] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:29.056358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:29.582692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:30.582040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:30.771750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:30.970476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:31.233261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:31.319729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:31.684812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:32.312729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:32.490712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:33.259160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:33.380164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:34.193227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:34.272492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.014650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.157645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.466423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.500648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.608980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.746046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:35.780532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:36.200002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:36.231533 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:36.500198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:36.552151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.076270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.168871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.265831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.494485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.660573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:37.758311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:38.031122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:39.637165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:41.241808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:43.143515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:43.145062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:43.207234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05a8c1-4a19-46a7-b1da-b4066b2dd7fe service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:43.212178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:43.213296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:43.213581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:43.213955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] PATH_INFO: `/v3/users/fba0678ca4ed484ca8c1c6bb75264076` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:43.215353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba0678ca4ed484ca8c1c6bb75264076)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:43.217438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:43.534535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba0678ca4ed484ca8c1c6bb75264076'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:43.535284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba0678ca4ed484ca8c1c6bb75264076'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:43.535966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba0678ca4ed484ca8c1c6bb75264076'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:43.556987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba0678ca4ed484ca8c1c6bb75264076'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:43.731645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba0678ca4ed484ca8c1c6bb75264076'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:43.744429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1028/2070] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:43 2022] DELETE /identity/v3/users/fba0678ca4ed484ca8c1c6bb75264076 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:43.749204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:43.749706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:43.749798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:43.767642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:43.768846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:43.827731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1043/2071] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:43.837452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:43.838881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:43.869637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:43.936875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8013b6-2560-42cf-a3da-82f64405109d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:43.940782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:43.942598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:43.942598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:43.942598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:43.951717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:43.953179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:43.963807 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1029/2072] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:44.693930 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:44.695432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:44.708126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb58fe-71e2-4ef9-98c0-3f97ff1baecd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:44.714013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:44.714981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:44.715267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:44.715542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] PATH_INFO: `/v3/projects/aed1573ca3ab4b8d83892f16ba1ff114` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:44.716796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed1573ca3ab4b8d83892f16ba1ff114)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:44.718829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:44.920996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed1573ca3ab4b8d83892f16ba1ff114', 'request_id': 'req-c1d7124e-06e3-4e6f-895d-231b9d03ece2'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:44.927253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed1573ca3ab4b8d83892f16ba1ff114', 'request_id': 'req-c1d7124e-06e3-4e6f-895d-231b9d03ece2'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.046404 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed1573ca3ab4b8d83892f16ba1ff114', 'request_id': 'req-c1d7124e-06e3-4e6f-895d-231b9d03ece2'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.059042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] The token cache is being invalidate because project aed1573ca3ab4b8d83892f16ba1ff114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:45.059336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.061421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.061643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1044/2073] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:44 2022] DELETE /identity/v3/projects/aed1573ca3ab4b8d83892f16ba1ff114 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:45.066203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:45.075909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a478345-ec82-4620-99cc-55843ececf7e service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:45.132212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:45.133111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:45.133392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:45.133664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] PATH_INFO: `/v3/users/752acb12d8fe4ac9805005a8ec53f220` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:45.134865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] RBAC: Authorizing `identity:delete_user(user_id=752acb12d8fe4ac9805005a8ec53f220)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:45.136808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:45.466415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752acb12d8fe4ac9805005a8ec53f220'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.467089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752acb12d8fe4ac9805005a8ec53f220'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.467707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752acb12d8fe4ac9805005a8ec53f220'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.480553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752acb12d8fe4ac9805005a8ec53f220'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.571957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8f6741c9-a4ef-47d7-a675-5b1975bb144f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752acb12d8fe4ac9805005a8ec53f220'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:45.585967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1030/2074] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:45 2022] DELETE /identity/v3/users/752acb12d8fe4ac9805005a8ec53f220 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:46.110460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:46.111960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:46.125705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7124e-06e3-4e6f-895d-231b9d03ece2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:46.186152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:46.187048 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:46.187940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:46.188434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] PATH_INFO: `/v3/projects/a863fd73477244cc893d203c6c9dfea1` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:46.189729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a863fd73477244cc893d203c6c9dfea1)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:46.191705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:46.202384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:46.326438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a863fd73477244cc893d203c6c9dfea1', 'request_id': 'req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:46.334680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a863fd73477244cc893d203c6c9dfea1', 'request_id': 'req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:46.696485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a863fd73477244cc893d203c6c9dfea1', 'request_id': 'req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:46.721035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] The token cache is being invalidate because project a863fd73477244cc893d203c6c9dfea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:46.721035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:46.721035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8cdedc9-3b06-4a36-93ff-136ac5c4b029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:46.721035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1045/2075] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:46 2022] DELETE /identity/v3/projects/a863fd73477244cc893d203c6c9dfea1 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:46.725757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016030bd-4896-48b7-b2fd-b0a6440fb25c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:46.726244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016030bd-4896-48b7-b2fd-b0a6440fb25c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:46.726244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016030bd-4896-48b7-b2fd-b0a6440fb25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:46.742309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:46.775282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-016030bd-4896-48b7-b2fd-b0a6440fb25c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:46.776614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-016030bd-4896-48b7-b2fd-b0a6440fb25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:46.856602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1031/2076] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:46.865345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f8c3-ce98-47ed-b284-e0024deb8395 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:46.865632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f8c3-ce98-47ed-b284-e0024deb8395 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:46.865909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f8c3-ce98-47ed-b284-e0024deb8395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:46.887450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8353f8c3-ce98-47ed-b284-e0024deb8395 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:46.888661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8353f8c3-ce98-47ed-b284-e0024deb8395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:46.897692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1046/2077] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:46.901924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016030bd-4896-48b7-b2fd-b0a6440fb25c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:46.912395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016030bd-4896-48b7-b2fd-b0a6440fb25c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:46.917147 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:46.917914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:46.918085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:46.918392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:46.919149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:46.919460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:46.922262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:46.935699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1032/2078] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:46.940814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f8c3-ce98-47ed-b284-e0024deb8395 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:46.955102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f8c3-ce98-47ed-b284-e0024deb8395 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:46.955346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:46.956130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:46.956472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:46.956817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:46.957623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:46.959664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:47.102378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1047/2079] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:46 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:57:47.104240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:47.118916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17685bf2-45c8-4dbf-9101-2ce6a2f39755 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:47.207429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:47.208034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:47.208319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:47.208596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:47.209290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:47.211271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:47.431215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1033/2080] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:47 2022] POST /identity/v3/users => generated 291 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:47.435541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:47.446702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe87d9-2570-4929-9944-8d75a5f5737b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:47.451123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:47.451771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:47.452052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:47.452324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:47.453490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:47.455123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:47.465725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1048/2081] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:47.469974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:47.480010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7077-39fa-4fe8-b018-7699c727d00c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:47.485982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aae932-1a9a-456e-927c-e890b6fe3213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:47.486748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aae932-1a9a-456e-927c-e890b6fe3213 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:47.487104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aae932-1a9a-456e-927c-e890b6fe3213 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:47.487446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aae932-1a9a-456e-927c-e890b6fe3213 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:47.488277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aae932-1a9a-456e-927c-e890b6fe3213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:47.489988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aae932-1a9a-456e-927c-e890b6fe3213 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:47.503520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1034/2082] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:47.506295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:47.519420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc333b-1022-4028-9622-11a9a32528bf admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:47.524702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:47.525713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:47.525999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:47.526312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] PATH_INFO: `/v3/projects/c603da2d6c7249c09ea26ce73859702f/users/9ab017fa5b8b4219b5dbe840b5e03497/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:47.544162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c603da2d6c7249c09ea26ce73859702f, user_id=9ab017fa5b8b4219b5dbe840b5e03497, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:47.557717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:47.720934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1049/2083] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:47 2022] PUT /identity/v3/projects/c603da2d6c7249c09ea26ce73859702f/users/9ab017fa5b8b4219b5dbe840b5e03497/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:47.732373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52195ec9-1208-4575-8a90-ff2e3dae7f35 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:47.732849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52195ec9-1208-4575-8a90-ff2e3dae7f35 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:47.732985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52195ec9-1208-4575-8a90-ff2e3dae7f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:47.760482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-52195ec9-1208-4575-8a90-ff2e3dae7f35 None None] MFA Rules not processed for user `9ab017fa5b8b4219b5dbe840b5e03497`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:47.764074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-52195ec9-1208-4575-8a90-ff2e3dae7f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:47.824757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1035/2084] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:47.829430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:47.892866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf6d07-5ad6-46fa-b156-f35bafaa5563 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:47.897631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:47.898459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:47.898620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:47.898901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:47.899596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:47.901520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:48.096327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1050/2085] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:47 2022] POST /identity/v3/projects => generated 381 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:48.100672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52195ec9-1208-4575-8a90-ff2e3dae7f35 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:48.113587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52195ec9-1208-4575-8a90-ff2e3dae7f35 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:48.171091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:48.172324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:48.172617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:48.172895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:48.173596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:48.175576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:48.379638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1036/2086] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:48 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:48.392638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:48.403349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9c49a-aa3e-4d01-8ed2-9d3fdad279e8 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:48.408311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:48.408998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:48.409360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:48.409565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:48.410353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:48.411999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:48.423891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1051/2087] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:48.428095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:48.439188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a4701-dce2-4786-a458-001d904c8483 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:48.444408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:48.445072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:48.445518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:48.445814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] PATH_INFO: `/v3/projects/e9833425ab3d4c3891afbce08a41df0b/users/2f445921af4840e9a841760ba84bf541/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:48.463949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9833425ab3d4c3891afbce08a41df0b, user_id=2f445921af4840e9a841760ba84bf541, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:48.465370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:48.578088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1037/2088] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:48 2022] PUT /identity/v3/projects/e9833425ab3d4c3891afbce08a41df0b/users/2f445921af4840e9a841760ba84bf541/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:48.582012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:48.592551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66326a8e-fb6e-44e0-b77d-e90d7528716b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:48.598246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:48.653817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ecd5f-e823-4794-b603-6bce65fe19b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:48.654523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ecd5f-e823-4794-b603-6bce65fe19b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:48.654811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ecd5f-e823-4794-b603-6bce65fe19b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:48.655131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ecd5f-e823-4794-b603-6bce65fe19b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:48.656833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ecd5f-e823-4794-b603-6bce65fe19b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:48.658496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ecd5f-e823-4794-b603-6bce65fe19b9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:48.668765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1052/2089] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:48.674274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:48.687056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eadf48-fcd6-44de-ab4c-52b13b858dff admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:48.694194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:48.695872 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:48.696297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:48.697072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] PATH_INFO: `/v3/projects/e9833425ab3d4c3891afbce08a41df0b/users/2f445921af4840e9a841760ba84bf541/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:48.700321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9833425ab3d4c3891afbce08a41df0b, user_id=2f445921af4840e9a841760ba84bf541, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:48.704963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:48.727592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1038/2090] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:48 2022] PUT /identity/v3/projects/e9833425ab3d4c3891afbce08a41df0b/users/2f445921af4840e9a841760ba84bf541/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:48.746900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff78005-fe11-4184-ad72-d82aa134e3d2 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:48.747215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff78005-fe11-4184-ad72-d82aa134e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:48.747498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff78005-fe11-4184-ad72-d82aa134e3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:48.776843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-dff78005-fe11-4184-ad72-d82aa134e3d2 None None] MFA Rules not processed for user `2f445921af4840e9a841760ba84bf541`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:48.781389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dff78005-fe11-4184-ad72-d82aa134e3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:48.854519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1053/2091] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:48.867029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:48.869607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:48.927229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8495c0a-b942-43c3-9993-3733780d5e4b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:48.930779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:48.931413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:48.931702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:48.931973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:48.941587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:48.943070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:48.952317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1039/2092] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:49.234296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff78005-fe11-4184-ad72-d82aa134e3d2 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:49.242809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff78005-fe11-4184-ad72-d82aa134e3d2 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:49.246270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d707c-b85b-464c-b286-043501154ea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:49.247024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d707c-b85b-464c-b286-043501154ea1 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:49.247544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d707c-b85b-464c-b286-043501154ea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:49.248045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d707c-b85b-464c-b286-043501154ea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:49.326327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d707c-b85b-464c-b286-043501154ea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:49.328279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d707c-b85b-464c-b286-043501154ea1 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:49.338089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1054/2093] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:49.410947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:50.004470 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:50.070148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1a187-34b7-4896-8fea-2c0913c14e12 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:50.074813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:50.075695 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:50.075987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:50.076256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] PATH_INFO: `/v3/users/9ab017fa5b8b4219b5dbe840b5e03497` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:50.077451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab017fa5b8b4219b5dbe840b5e03497)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:50.079601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:50.507167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab017fa5b8b4219b5dbe840b5e03497'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:50.507831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab017fa5b8b4219b5dbe840b5e03497'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:50.508439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab017fa5b8b4219b5dbe840b5e03497'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:50.522718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab017fa5b8b4219b5dbe840b5e03497'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:50.759240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab017fa5b8b4219b5dbe840b5e03497'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:50.773252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1040/2094] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:50 2022] DELETE /identity/v3/users/9ab017fa5b8b4219b5dbe840b5e03497 => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:50.777858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:50.778370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:50.778471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:50.798031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:50.799266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:50.863680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1055/2095] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:50.873893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:50.875320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:50.934026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e0838-d6c3-4447-916d-eecf2ca018e9 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:50.938389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:50.939035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:50.939316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:50.939590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:50.948617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:50.950112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:50.960170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1041/2096] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:51.538755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:51.540198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:51.558135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c4152-f5f5-4d0d-8ca1-76911e33b9ef None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:51.562863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:51.563922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:51.564365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:51.565640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] PATH_INFO: `/v3/projects/c603da2d6c7249c09ea26ce73859702f` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:51.567344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c603da2d6c7249c09ea26ce73859702f)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:51.569336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:51.813535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c603da2d6c7249c09ea26ce73859702f', 'request_id': 'req-60b01ae1-d48d-4f98-bb21-da95ef05aa70'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:51.819912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c603da2d6c7249c09ea26ce73859702f', 'request_id': 'req-60b01ae1-d48d-4f98-bb21-da95ef05aa70'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:51.987561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c603da2d6c7249c09ea26ce73859702f', 'request_id': 'req-60b01ae1-d48d-4f98-bb21-da95ef05aa70'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.000209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] The token cache is being invalidate because project c603da2d6c7249c09ea26ce73859702f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:52.000506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.001189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.002886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1056/2097] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:51 2022] DELETE /identity/v3/projects/c603da2d6c7249c09ea26ce73859702f => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:52.006913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:52.017346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6e6cd-9068-4f77-9384-8927682c76ef service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:52.075649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:52.076440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:52.076973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:52.076973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] PATH_INFO: `/v3/users/2f445921af4840e9a841760ba84bf541` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:52.078423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f445921af4840e9a841760ba84bf541)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:52.080381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:52.415929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:52.524993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f445921af4840e9a841760ba84bf541'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.524993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f445921af4840e9a841760ba84bf541'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.524993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f445921af4840e9a841760ba84bf541'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.534578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f445921af4840e9a841760ba84bf541'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.754719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-196f8990-44f7-44d2-b9b9-a8f4a9486fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f445921af4840e9a841760ba84bf541'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:52.771349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1042/2098] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:57:52 2022] DELETE /identity/v3/users/2f445921af4840e9a841760ba84bf541 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:52.959340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:53.269895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:53.271384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:53.284564 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b01ae1-d48d-4f98-bb21-da95ef05aa70 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:53.342152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:53.342927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:53.343210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:53.343487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] PATH_INFO: `/v3/projects/e9833425ab3d4c3891afbce08a41df0b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:53.344690 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9833425ab3d4c3891afbce08a41df0b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:53.346655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:53.513285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9833425ab3d4c3891afbce08a41df0b', 'request_id': 'req-201088fc-4806-42ee-b3d9-6dadfc42f75f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:53.528169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9833425ab3d4c3891afbce08a41df0b', 'request_id': 'req-201088fc-4806-42ee-b3d9-6dadfc42f75f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:53.678366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9833425ab3d4c3891afbce08a41df0b', 'request_id': 'req-201088fc-4806-42ee-b3d9-6dadfc42f75f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:53.689971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] The token cache is being invalidate because project e9833425ab3d4c3891afbce08a41df0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:57:53.690294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:53.690935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-201088fc-4806-42ee-b3d9-6dadfc42f75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:57:53.693022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1057/2099] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:57:53 2022] DELETE /identity/v3/projects/e9833425ab3d4c3891afbce08a41df0b => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:53.697651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998e72-6bc9-429d-ae9a-f2f1598d39a9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:53.698148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998e72-6bc9-429d-ae9a-f2f1598d39a9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:53.698148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998e72-6bc9-429d-ae9a-f2f1598d39a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:53.716465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6c998e72-6bc9-429d-ae9a-f2f1598d39a9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:53.718467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6c998e72-6bc9-429d-ae9a-f2f1598d39a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:53.782209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1043/2100] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:53.790130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b7358-a2b2-4625-86e5-2b7a2b535595 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:53.790440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b7358-a2b2-4625-86e5-2b7a2b535595 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:53.790717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b7358-a2b2-4625-86e5-2b7a2b535595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:53.816165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2b3b7358-a2b2-4625-86e5-2b7a2b535595 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:53.818118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2b3b7358-a2b2-4625-86e5-2b7a2b535595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:53.830320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1058/2101] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:53.833428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998e72-6bc9-429d-ae9a-f2f1598d39a9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:53.857749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998e72-6bc9-429d-ae9a-f2f1598d39a9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:53.862935 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:53.863599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:53.870362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:53.870362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:53.870362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:57:53.870362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:53.870362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:53.879749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1044/2102] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:57:53.884110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b7358-a2b2-4625-86e5-2b7a2b535595 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:53.894273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b7358-a2b2-4625-86e5-2b7a2b535595 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:53.898957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:53.899615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:53.899898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:53.900174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:53.900878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:53.902990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:54.110207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1059/2103] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:53 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:54.113629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:54.129157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f2519-6461-40a9-988f-707a39298f31 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:54.183083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:54.183760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:54.184041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:54.184314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:54.185007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:54.187213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:54.250004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:54.355123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1045/2104] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:54 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:54.356361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:54.367062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd0c8b-68c9-4ba6-afef-0de2f76feff7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:54.371815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:54.372471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:54.374493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:54.374785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:54.375520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:54.377125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:54.387384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1060/2105] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:54.392350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:54.403845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf545c3-466f-4eaa-8832-b2be0262cbf3 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:54.408644 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4627c-e213-49b1-be21-f8744fe3a9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:54.409346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4627c-e213-49b1-be21-f8744fe3a9ca admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:54.409631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4627c-e213-49b1-be21-f8744fe3a9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:54.409910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4627c-e213-49b1-be21-f8744fe3a9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:54.410696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4627c-e213-49b1-be21-f8744fe3a9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:54.412371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4627c-e213-49b1-be21-f8744fe3a9ca admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:54.422907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1046/2106] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:54.427162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:54.438917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d9b9e-6bdd-4ce4-b1b4-2eb352d9bb41 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:54.443652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:54.444539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:54.444824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:54.445105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] PATH_INFO: `/v3/projects/1cc06e13e15b4a82a504c51c671de2d5/users/81bd2b29dda245b5880642145c85b39d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:54.459990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc06e13e15b4a82a504c51c671de2d5, user_id=81bd2b29dda245b5880642145c85b39d, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:54.462809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:54.524482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:54.606130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1061/2107] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:54 2022] PUT /identity/v3/projects/1cc06e13e15b4a82a504c51c671de2d5/users/81bd2b29dda245b5880642145c85b39d/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:54.617453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287d275-3a7d-4766-93a1-474a8a8a43fb None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:54.617764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287d275-3a7d-4766-93a1-474a8a8a43fb None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:54.618044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287d275-3a7d-4766-93a1-474a8a8a43fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:54.647904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9287d275-3a7d-4766-93a1-474a8a8a43fb None None] MFA Rules not processed for user `81bd2b29dda245b5880642145c85b39d`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:54.649173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9287d275-3a7d-4766-93a1-474a8a8a43fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:54.721617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1047/2108] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:57:54.725941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:54.789392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfc69b-24a9-4282-a734-e0b8fbf79a2b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:54.794012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:54.795227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:54.795515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:54.795824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:54.796611 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:54.798452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:54.975109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1062/2109] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:57:54 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:54.976859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287d275-3a7d-4766-93a1-474a8a8a43fb None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:55.000829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287d275-3a7d-4766-93a1-474a8a8a43fb None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:55.062209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:55.062891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:55.063343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:55.063622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:55.064320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:55.066325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:55.233890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1048/2110] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:57:54 2022] POST /identity/v3/users => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:57:55.237846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:55.248374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e65e8-b951-473a-9f80-09e638d579bb admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:55.253555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:55.254403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:55.254740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:55.255020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:55.255763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:55.257430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:55.269688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1063/2111] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:57:55.273800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:55.284556 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc004c1-f5d9-424f-8656-c28bc7434883 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:55.289523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:55.290444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:55.290728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:55.290999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] PATH_INFO: `/v3/projects/6adf0baa6d8d40eea1f0895e340a64c9/users/545db6358e4e47938e813d8f34ccf491/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:55.308043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6adf0baa6d8d40eea1f0895e340a64c9, user_id=545db6358e4e47938e813d8f34ccf491, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:55.311067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:55.438765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1049/2112] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:55 2022] PUT /identity/v3/projects/6adf0baa6d8d40eea1f0895e340a64c9/users/545db6358e4e47938e813d8f34ccf491/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:57:55.442997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:55.453783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5eab1-1d8f-418e-b8f1-7452a1f2dff1 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:55.511834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bae1d-364f-4834-bb05-02a18bda056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:55.512500 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bae1d-364f-4834-bb05-02a18bda056c admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:55.512783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bae1d-364f-4834-bb05-02a18bda056c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:55.513093 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bae1d-364f-4834-bb05-02a18bda056c admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:55.513818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bae1d-364f-4834-bb05-02a18bda056c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:55.515450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736bae1d-364f-4834-bb05-02a18bda056c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:55.525273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1064/2113] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:57:55.529694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:55.540674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b7d30-f037-4186-8142-788d14d35275 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:55.546185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:55.547091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:55.547378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:55.547657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] PATH_INFO: `/v3/projects/6adf0baa6d8d40eea1f0895e340a64c9/users/545db6358e4e47938e813d8f34ccf491/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:55.550054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6adf0baa6d8d40eea1f0895e340a64c9, user_id=545db6358e4e47938e813d8f34ccf491, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:55.553332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:55.567373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1050/2114] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:57:55 2022] PUT /identity/v3/projects/6adf0baa6d8d40eea1f0895e340a64c9/users/545db6358e4e47938e813d8f34ccf491/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:57:55.578396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbed49e-081b-44a5-ab89-cee0abcfef0a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:55.578847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbed49e-081b-44a5-ab89-cee0abcfef0a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:55.579128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbed49e-081b-44a5-ab89-cee0abcfef0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:55.609422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5dbed49e-081b-44a5-ab89-cee0abcfef0a None None] MFA Rules not processed for user `545db6358e4e47938e813d8f34ccf491`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:57:55.610267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5dbed49e-081b-44a5-ab89-cee0abcfef0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:55.696312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1065/2115] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:57:55.712851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:57:55.715562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:57:55.791712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e823e-edff-411d-84f0-1ac50d079e94 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:57:55.795833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:57:55.796995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:57:55.796995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:57:55.797448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:57:55.894279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:57:55.895903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:57:55.908721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1051/2116] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:57:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:57:55.954067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:56.062144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:57:57.952530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:02.076400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e04c-e062-42be-a73b-4978246cd26c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:02.077304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e04c-e062-42be-a73b-4978246cd26c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:02.077586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e04c-e062-42be-a73b-4978246cd26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:02.108187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-19d6e04c-e062-42be-a73b-4978246cd26c None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:02.109517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-19d6e04c-e062-42be-a73b-4978246cd26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:02.169367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1066/2117] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:02.219132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:02.220618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:02.306718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d30cc-adc3-4b7b-b99e-64b6db99cc5f service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:02.312952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:02.320600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:02.320600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:02.320600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:02.323355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:02.325887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:02.334094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1052/2118] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:58:03.335311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a732b-8b94-478d-96fc-c2b6528a6dd2 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:03.335839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a732b-8b94-478d-96fc-c2b6528a6dd2 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:03.335952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a732b-8b94-478d-96fc-c2b6528a6dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:03.355211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b07a732b-8b94-478d-96fc-c2b6528a6dd2 None None] MFA Rules not processed for user `cbc9d5ce4a3241ea898f216543114fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:03.356471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b07a732b-8b94-478d-96fc-c2b6528a6dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:03.418677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:03.425151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1067/2119] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:03.425151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d484-4f0b-465d-8680-18b48e4032ff None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:03.425151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d484-4f0b-465d-8680-18b48e4032ff None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:03.425151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d484-4f0b-465d-8680-18b48e4032ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:03.429593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a98b6-815c-43b2-a2ea-66c63a77ed33 service swift] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:03.434527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bc05f-182c-49f4-826b-b5c8eca3c040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:03.435335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bc05f-182c-49f4-826b-b5c8eca3c040 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:03.435615 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bc05f-182c-49f4-826b-b5c8eca3c040 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:03.435892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bc05f-182c-49f4-826b-b5c8eca3c040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:03.442909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8075d484-4f0b-465d-8680-18b48e4032ff None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:03.444317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8075d484-4f0b-465d-8680-18b48e4032ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:03.446283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bc05f-182c-49f4-826b-b5c8eca3c040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:03.447730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bc05f-182c-49f4-826b-b5c8eca3c040 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:03.452471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1068/2120] 10.222.0.23 () {58 vars in 967 bytes} [Wed Jun 15 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:03.458208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1053/2121] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:03.467226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d484-4f0b-465d-8680-18b48e4032ff None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:03.481530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d484-4f0b-465d-8680-18b48e4032ff None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:03.488637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:03.488637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:03.488637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:03.488637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:03.496327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:03.499117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:03.501132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:03.511645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1069/2122] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:58:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:58:04.475458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3143df8-7270-4d67-adb2-539ac03dfee3 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:04.475971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3143df8-7270-4d67-adb2-539ac03dfee3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:04.476079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3143df8-7270-4d67-adb2-539ac03dfee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:04.495580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f3143df8-7270-4d67-adb2-539ac03dfee3 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:04.496706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f3143df8-7270-4d67-adb2-539ac03dfee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:04.505575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1054/2123] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:04.520866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:04.545137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1a4a2-8720-4a38-aad4-b1f6fe5c75c2 service swift] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:04.558319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:04.559103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:04.559299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:04.559577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:04.569694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:04.571773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:04.581143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1070/2124] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:58:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:58:05.691834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:05.692296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:05.692676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:05.717738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:05.719225 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:05.726683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1055/2125] 10.222.0.23 () {58 vars in 967 bytes} [Wed Jun 15 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:05.739280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:05.751654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b02bad-60c8-41e4-92ab-21e292101386 service swift] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:05.759518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:05.759518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:05.759518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:05.759518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:05.771389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:05.774111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:05.793383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1071/2126] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 14:58:09.642750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:09.644225 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:09.694313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:09.757928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432f5d-83b2-4179-afb2-4006ad7cbbe9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:09.766112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:09.767123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:09.767613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:09.767889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] PATH_INFO: `/v3/users/8a74a23b13e24616921c55bc6226216f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:09.769806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a74a23b13e24616921c55bc6226216f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:09.772932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:10.170550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a74a23b13e24616921c55bc6226216f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.171308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a74a23b13e24616921c55bc6226216f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.171994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a74a23b13e24616921c55bc6226216f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.188737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a74a23b13e24616921c55bc6226216f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.256716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:10.268323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca55d16-d740-4930-841f-b564542a5d71 service swift] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:10.273235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:10.274110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:10.274440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:10.274717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] PATH_INFO: `/v3/users/81bd2b29dda245b5880642145c85b39d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:10.276040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=81bd2b29dda245b5880642145c85b39d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:10.277978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:10.293080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-2ea0abcd-985e-4e28-8081-1f8d014753ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a74a23b13e24616921c55bc6226216f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.308577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1056/2127] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:58:09 2022] DELETE /identity/v3/users/8a74a23b13e24616921c55bc6226216f => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:10.312783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:10.313244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:10.313623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:10.332283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:10.333770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:10.399680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1057/2128] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:10.411024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:10.412395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:10.476112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:10.478698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184ced7-aabc-4dcd-902b-36696cf5f1c0 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:10.482990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f30e1d-4f59-49b8-8529-73ffdeb817d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:10.483585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f30e1d-4f59-49b8-8529-73ffdeb817d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:10.483862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f30e1d-4f59-49b8-8529-73ffdeb817d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:10.484132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f30e1d-4f59-49b8-8529-73ffdeb817d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:10.494554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f30e1d-4f59-49b8-8529-73ffdeb817d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:10.496102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f30e1d-4f59-49b8-8529-73ffdeb817d2 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:10.506264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1058/2129] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:58:10.535682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81bd2b29dda245b5880642145c85b39d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.536459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81bd2b29dda245b5880642145c85b39d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.545055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81bd2b29dda245b5880642145c85b39d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.708658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81bd2b29dda245b5880642145c85b39d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.723321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81bd2b29dda245b5880642145c85b39d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:10.730582 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1072/2130] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:58:10 2022] DELETE /identity/v3/users/81bd2b29dda245b5880642145c85b39d => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:58:10.735225 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:10.735744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:10.735744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:10.755842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:10.757072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:10.813998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:10.816368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:10.835129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1059/2131] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:10.843317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:10.845044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:10.851020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f1210-df1a-4325-b6ba-4a8c8296b5da admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:10.867700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:10.869298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:10.869473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:10.870477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] PATH_INFO: `/v3/projects/22a6143a5b05471e8aed517230f734e7` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:10.874010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a6143a5b05471e8aed517230f734e7)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:10.882367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:10.950630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06aa528-2c34-4df1-ba28-00a4a0cc0702 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:10.956420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82824263-61fc-493e-b5cd-b17d54c0c479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:10.957262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824263-61fc-493e-b5cd-b17d54c0c479 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:10.957652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824263-61fc-493e-b5cd-b17d54c0c479 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:10.958029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824263-61fc-493e-b5cd-b17d54c0c479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:10.973007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82824263-61fc-493e-b5cd-b17d54c0c479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:10.977378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82824263-61fc-493e-b5cd-b17d54c0c479 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:11.008273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1060/2132] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:11.079391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a6143a5b05471e8aed517230f734e7', 'request_id': 'req-694cc470-5649-4cf0-b65b-08672b25572d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:11.096293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a6143a5b05471e8aed517230f734e7', 'request_id': 'req-694cc470-5649-4cf0-b65b-08672b25572d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:11.292674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a6143a5b05471e8aed517230f734e7', 'request_id': 'req-694cc470-5649-4cf0-b65b-08672b25572d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:11.312031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] The token cache is being invalidate because project 22a6143a5b05471e8aed517230f734e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:58:11.312702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:11.313991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-694cc470-5649-4cf0-b65b-08672b25572d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:11.316161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1073/2133] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:58:10 2022] DELETE /identity/v3/projects/22a6143a5b05471e8aed517230f734e7 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:58:11.321208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65cfa-091c-486a-8936-b0c29772d50e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:11.321536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65cfa-091c-486a-8936-b0c29772d50e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:11.322030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65cfa-091c-486a-8936-b0c29772d50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:11.356077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1ec65cfa-091c-486a-8936-b0c29772d50e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:11.360944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1ec65cfa-091c-486a-8936-b0c29772d50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:11.454524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1061/2134] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:11.461048 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f7d38-87d4-48a0-a840-6ef4b118a2ec None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:11.461334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f7d38-87d4-48a0-a840-6ef4b118a2ec None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:11.461607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f7d38-87d4-48a0-a840-6ef4b118a2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:11.484502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-cd3f7d38-87d4-48a0-a840-6ef4b118a2ec None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:11.485724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cd3f7d38-87d4-48a0-a840-6ef4b118a2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:11.494871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1074/2135] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:11.503978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65cfa-091c-486a-8936-b0c29772d50e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:11.514416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65cfa-091c-486a-8936-b0c29772d50e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:11.521493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:11.522554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:11.522984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:11.523403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:11.524601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:58:11.529692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:11.529692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:11.541913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1062/2136] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:58:11.546460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f7d38-87d4-48a0-a840-6ef4b118a2ec None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:11.562661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f7d38-87d4-48a0-a840-6ef4b118a2ec None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:11.567439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:11.568106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:11.568382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:11.568652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:11.569355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:11.571304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:11.589280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:11.758280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:11.761911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:11.762059 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1075/2137] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:58:11 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:11.763135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:11.778440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f9e29-5be5-4e90-911d-9cf613c7056f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:11.872504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b129f-d083-4bf5-86e9-a93f9b0c87ce admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:11.878562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:11.879541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:11.879895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:11.880359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:11.881323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:11.883748 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:11.887160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:11.888540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:11.889028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:11.889692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] PATH_INFO: `/v3/projects/1cc06e13e15b4a82a504c51c671de2d5` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:11.891504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc06e13e15b4a82a504c51c671de2d5)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:11.893712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:11.962096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:12.029459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1076/2138] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:58:11 2022] POST /identity/v3/users => generated 287 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:12.030672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:12.045878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099be5-b7a1-410e-8426-19245abd26da admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:12.051509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc06e13e15b4a82a504c51c671de2d5', 'request_id': 'req-af56db06-c916-4a16-b7f4-217b9bfafb1b'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.120545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:12.140425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:12.140425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:12.140425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:12.140425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:12.140425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:12.140425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1077/2139] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:58:12.140883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:12.151114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaa83d-4d81-4e24-af09-d1de50b554bd admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:12.156084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:12.156817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:12.157684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:12.157684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:12.158322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:12.159976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:12.176374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1078/2140] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:58:12.182721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc06e13e15b4a82a504c51c671de2d5', 'request_id': 'req-af56db06-c916-4a16-b7f4-217b9bfafb1b'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.182721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc06e13e15b4a82a504c51c671de2d5', 'request_id': 'req-af56db06-c916-4a16-b7f4-217b9bfafb1b'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.184277 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:12.197364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] The token cache is being invalidate because project 1cc06e13e15b4a82a504c51c671de2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:58:12.197965 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.198107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd4b53-caa8-469c-8da7-2127ad0be18e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:12.198911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-af56db06-c916-4a16-b7f4-217b9bfafb1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.203375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1063/2141] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:58:11 2022] DELETE /identity/v3/projects/1cc06e13e15b4a82a504c51c671de2d5 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:12.203375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:12.203375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:12.203375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:12.205516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8acc3-54e9-403d-8b58-eedcc3dc3827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:12.206512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8acc3-54e9-403d-8b58-eedcc3dc3827 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:12.206883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8acc3-54e9-403d-8b58-eedcc3dc3827 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:12.207200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8acc3-54e9-403d-8b58-eedcc3dc3827 admin admin] PATH_INFO: `/v3/projects/06d1766d477e4d3993d1474d6e1a30e6/users/26608323a21f4cb7832b93bdfd971663/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:12.220904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] MFA Rules not processed for user `e367e391e68a477d861a515f846a8dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:12.222595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:12.232083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8acc3-54e9-403d-8b58-eedcc3dc3827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d1766d477e4d3993d1474d6e1a30e6, user_id=26608323a21f4cb7832b93bdfd971663, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:12.235190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8acc3-54e9-403d-8b58-eedcc3dc3827 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:12.281933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1064/2142] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:12.282817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:12.284208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:12.299547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350aa0a1-be57-4e7e-bfc5-d5297c93d338 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:12.307215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:12.308643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:12.308936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:12.309217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] PATH_INFO: `/v3/users/545db6358e4e47938e813d8f34ccf491` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:12.310949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=545db6358e4e47938e813d8f34ccf491)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:12.312911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:12.347033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1079/2143] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:58:12 2022] PUT /identity/v3/projects/06d1766d477e4d3993d1474d6e1a30e6/users/26608323a21f4cb7832b93bdfd971663/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:12.349071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:12.349354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:12.349633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:12.368080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:12.369889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:12.437485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1080/2144] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:12.447980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:12.449394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:12.510991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905099d7-816b-4f8a-9a8e-ac4a79350c44 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:12.514777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:12.515432 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:12.515711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:12.516001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:12.531875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:12.533363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:12.543890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1081/2145] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:58:12.626658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:12.629817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '545db6358e4e47938e813d8f34ccf491'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.630910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '545db6358e4e47938e813d8f34ccf491'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.631708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '545db6358e4e47938e813d8f34ccf491'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.646908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '545db6358e4e47938e813d8f34ccf491'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.789689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01aa1b32-85fb-4b2e-bfa5-399a9385d594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '545db6358e4e47938e813d8f34ccf491'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:12.803863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1065/2146] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:58:12 2022] DELETE /identity/v3/users/545db6358e4e47938e813d8f34ccf491 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:13.457102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:13.467418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:13.479113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c430c-7d63-4122-90fc-f23dd8721831 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:13.560078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:13.560961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:13.561270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:13.561711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] PATH_INFO: `/v3/projects/6adf0baa6d8d40eea1f0895e340a64c9` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:13.569224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6adf0baa6d8d40eea1f0895e340a64c9)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:13.571905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:13.725990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6adf0baa6d8d40eea1f0895e340a64c9', 'request_id': 'req-faa51d7a-0d82-469f-8356-6584b63d9d2b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:13.732523 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6adf0baa6d8d40eea1f0895e340a64c9', 'request_id': 'req-faa51d7a-0d82-469f-8356-6584b63d9d2b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:13.833997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6adf0baa6d8d40eea1f0895e340a64c9', 'request_id': 'req-faa51d7a-0d82-469f-8356-6584b63d9d2b'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:13.849076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] The token cache is being invalidate because project 6adf0baa6d8d40eea1f0895e340a64c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:58:13.849479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:13.850543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-faa51d7a-0d82-469f-8356-6584b63d9d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:13.858749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1082/2147] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:58:13 2022] DELETE /identity/v3/projects/6adf0baa6d8d40eea1f0895e340a64c9 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:13.860346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926868ac-0d96-4c82-b22d-e7606a5c45e0 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:13.861302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926868ac-0d96-4c82-b22d-e7606a5c45e0 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:13.861590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926868ac-0d96-4c82-b22d-e7606a5c45e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:13.887086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-926868ac-0d96-4c82-b22d-e7606a5c45e0 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:13.888334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-926868ac-0d96-4c82-b22d-e7606a5c45e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:13.950875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1066/2148] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:13.959256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f730-cab2-4e17-9b55-e84b455546fe None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:13.959574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f730-cab2-4e17-9b55-e84b455546fe None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:13.959848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f730-cab2-4e17-9b55-e84b455546fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:13.978899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-57d7f730-cab2-4e17-9b55-e84b455546fe None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:13.980094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-57d7f730-cab2-4e17-9b55-e84b455546fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:13.988113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1083/2149] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:13.992609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926868ac-0d96-4c82-b22d-e7606a5c45e0 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.002761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926868ac-0d96-4c82-b22d-e7606a5c45e0 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:14.007519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:14.008183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.008505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.009310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.010049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:58:14.010568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:14.012194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:14.025451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1067/2150] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:58:14.029832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f730-cab2-4e17-9b55-e84b455546fe None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.039709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f730-cab2-4e17-9b55-e84b455546fe None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:14.044676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:14.045478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.045757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.046319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.047159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:14.049224 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:14.252948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1084/2151] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:58:14 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:58:14.257332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.268689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398999e0-8fd0-4e38-9a33-7d2ad3da3c4c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:14.355630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:14.357116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.357478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.357919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.358885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:14.361089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:14.504367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1068/2152] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:58:14 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:14.510456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.532338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e39d3-ad7a-45ab-b071-5a521d79b3b8 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:14.539281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:14.540497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.541164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.541705 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.542716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:14.544571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:14.558986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1085/2153] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:58:14.563052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.574809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88d8bd-44eb-46f4-acee-a5dc6f85e3ae admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:14.594610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b28b9-3515-490f-81e5-687c93176376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:14.595303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b28b9-3515-490f-81e5-687c93176376 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.595971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b28b9-3515-490f-81e5-687c93176376 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.595971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b28b9-3515-490f-81e5-687c93176376 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.596690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b28b9-3515-490f-81e5-687c93176376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:14.598350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b28b9-3515-490f-81e5-687c93176376 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:14.611496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1069/2154] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:58:14.615869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.627896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611d1e0-5cdd-44b4-b2c3-c64666a588a8 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:14.633695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:14.634982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.635445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.635866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] PATH_INFO: `/v3/projects/a2836c0ab903447597888f1b062ed0f1/users/2490aeaaa8e842c0be089e0b744a59d9/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.654105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2836c0ab903447597888f1b062ed0f1, user_id=2490aeaaa8e842c0be089e0b744a59d9, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:14.657683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:14.819607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1086/2155] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:58:14 2022] PUT /identity/v3/projects/a2836c0ab903447597888f1b062ed0f1/users/2490aeaaa8e842c0be089e0b744a59d9/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:14.820921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c863eb2-1d2d-4d51-949d-dabab9bab2c1 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:14.821300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c863eb2-1d2d-4d51-949d-dabab9bab2c1 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:14.821677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c863eb2-1d2d-4d51-949d-dabab9bab2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:14.873766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-8c863eb2-1d2d-4d51-949d-dabab9bab2c1 None None] MFA Rules not processed for user `2490aeaaa8e842c0be089e0b744a59d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:14.875738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8c863eb2-1d2d-4d51-949d-dabab9bab2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:14.891128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:14.955849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1070/2156] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:14.969297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:14.971151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:15.034873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:15.038800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93671c-2375-40c3-9b0c-ac5e7e183fcb admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:15.054210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:15.054210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:15.054210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:15.054210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:15.055087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:15.057317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:15.068164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1087/2157] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:16.438467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de364aef-ffd8-4d21-8d23-c02419db6e1e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:16.438985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de364aef-ffd8-4d21-8d23-c02419db6e1e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:16.439096 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de364aef-ffd8-4d21-8d23-c02419db6e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:16.479188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-de364aef-ffd8-4d21-8d23-c02419db6e1e None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:16.480889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-de364aef-ffd8-4d21-8d23-c02419db6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:16.594593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1071/2158] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:16.609365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:16.611373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:16.707651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be1d2d-ef04-43eb-be77-b7c873355db6 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:16.714683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:16.715430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:16.715715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:16.715987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:16.725270 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:16.728299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:16.750474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1088/2159] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:58:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:58:21.003336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfae164-e3e0-4e72-be90-581c0d30d8bc None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:21.003877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfae164-e3e0-4e72-be90-581c0d30d8bc None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:21.004089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfae164-e3e0-4e72-be90-581c0d30d8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:21.074627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3dfae164-e3e0-4e72-be90-581c0d30d8bc None None] MFA Rules not processed for user `26608323a21f4cb7832b93bdfd971663`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:21.080080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3dfae164-e3e0-4e72-be90-581c0d30d8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:21.190447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1072/2160] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:21.191998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:21.194073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:21.286888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:21.330579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa660ff-ce59-4462-ae1e-23c45d5cadc4 service swift] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:21.338366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:21.339441 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:21.339730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:21.340004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:21.341752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:21.343390 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:21.510740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1089/2161] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:58:21 2022] POST /identity/v3/projects => generated 373 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:21.514907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfae164-e3e0-4e72-be90-581c0d30d8bc None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:21.529730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfae164-e3e0-4e72-be90-581c0d30d8bc None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:21.608286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:21.631477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:21.632662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:21.632954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:21.638536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:21.638536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:21.638536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:22.315030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1073/2162] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:58:21 2022] POST /identity/v3/users => generated 293 bytes in 801 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:22.319411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:22.333339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add0295-4206-4d52-8fca-e72d3a117c9e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:22.340647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:22.341655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:22.342024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:22.342412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:22.343225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:22.345244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:22.384138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1090/2163] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 14:58:22.388220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:22.400596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a4739-a405-408c-ab99-5355e3d3b0de admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:22.408147 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:22.409172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:22.409647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:22.409934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] PATH_INFO: `/v3/projects/be09bb633e2549568d645fbfcf6aafa5/users/dfd0e1ae917d49c8839c36c50d41217d/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:22.430162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be09bb633e2549568d645fbfcf6aafa5, user_id=dfd0e1ae917d49c8839c36c50d41217d, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:22.433299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:22.612949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1074/2164] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:58:22 2022] PUT /identity/v3/projects/be09bb633e2549568d645fbfcf6aafa5/users/dfd0e1ae917d49c8839c36c50d41217d/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:22.617486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:22.629602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67311b96-44e7-4778-afdc-4bcd24d7b16e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:22.762222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff405cc-7c38-42f6-9798-92202157a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:22.762222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff405cc-7c38-42f6-9798-92202157a52f admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:22.762222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff405cc-7c38-42f6-9798-92202157a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:22.762222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff405cc-7c38-42f6-9798-92202157a52f admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:22.763096 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff405cc-7c38-42f6-9798-92202157a52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:22.763096 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff405cc-7c38-42f6-9798-92202157a52f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:22.776716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1091/2165] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:58:22.780758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:22.792328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be64b-02d4-4d37-9c92-f09263951e9d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:22.799343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:22.800923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:22.801211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:22.801487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] PATH_INFO: `/v3/projects/be09bb633e2549568d645fbfcf6aafa5/users/dfd0e1ae917d49c8839c36c50d41217d/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:22.805051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be09bb633e2549568d645fbfcf6aafa5, user_id=dfd0e1ae917d49c8839c36c50d41217d, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:22.812383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:22.836551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1075/2166] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:58:22 2022] PUT /identity/v3/projects/be09bb633e2549568d645fbfcf6aafa5/users/dfd0e1ae917d49c8839c36c50d41217d/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:32.733087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e51a-1ddf-4f2f-9913-5f24114977ea None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:32.733087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e51a-1ddf-4f2f-9913-5f24114977ea None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:32.733087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e51a-1ddf-4f2f-9913-5f24114977ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:32.777661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4c90e51a-1ddf-4f2f-9913-5f24114977ea None None] MFA Rules not processed for user `dfd0e1ae917d49c8839c36c50d41217d`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:32.779505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4c90e51a-1ddf-4f2f-9913-5f24114977ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:32.914401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1092/2167] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:32.945787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:32.947547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:33.112284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6ae3-9261-4a08-a482-35c18291bca2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:33.119618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:33.124400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:33.124400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:33.124400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:33.234771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:33.239758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:33.251170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1076/2168] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:38.219465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dfe49-83bc-4dda-811f-c2ee4ec2b4b1 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:38.219955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dfe49-83bc-4dda-811f-c2ee4ec2b4b1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:38.220072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dfe49-83bc-4dda-811f-c2ee4ec2b4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:38.240107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-321dfe49-83bc-4dda-811f-c2ee4ec2b4b1 None None] MFA Rules not processed for user `26608323a21f4cb7832b93bdfd971663`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:38.242160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-321dfe49-83bc-4dda-811f-c2ee4ec2b4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:38.255013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1093/2169] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:43.245524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:48.233532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:49.985205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:49.987231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:50.106859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2c55d-ab0f-40cb-b476-0b15ee3f8a90 service nova] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:50.113134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adba3ba-7cca-440f-8033-a593c2ce9738 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:50.113624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba3ba-7cca-440f-8033-a593c2ce9738 service placement] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:50.113913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba3ba-7cca-440f-8033-a593c2ce9738 service placement] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:50.114207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba3ba-7cca-440f-8033-a593c2ce9738 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:50.116425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1adba3ba-7cca-440f-8033-a593c2ce9738 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:50.128221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adba3ba-7cca-440f-8033-a593c2ce9738 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:50.130941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adba3ba-7cca-440f-8033-a593c2ce9738 service placement] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:50.141995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1077/2170] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:52.788823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dfe49-83bc-4dda-811f-c2ee4ec2b4b1 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:52.860560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dfe49-83bc-4dda-811f-c2ee4ec2b4b1 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:52.865313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:52.866246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:52.866533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:52.866813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] PATH_INFO: `/v3/users/2490aeaaa8e842c0be089e0b744a59d9` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:52.868058 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2490aeaaa8e842c0be089e0b744a59d9)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:52.870013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:53.169029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2490aeaaa8e842c0be089e0b744a59d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:53.169839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2490aeaaa8e842c0be089e0b744a59d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:53.181955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2490aeaaa8e842c0be089e0b744a59d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:53.312151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2490aeaaa8e842c0be089e0b744a59d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:53.326079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2490aeaaa8e842c0be089e0b744a59d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:53.333002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1094/2171] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:58:52 2022] DELETE /identity/v3/users/2490aeaaa8e842c0be089e0b744a59d9 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:53.337548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9e3d2-6f79-4aa2-8471-a2348767724b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:53.338058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9e3d2-6f79-4aa2-8471-a2348767724b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:53.338154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9e3d2-6f79-4aa2-8471-a2348767724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:53.355765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e4c9e3d2-6f79-4aa2-8471-a2348767724b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:53.356995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e4c9e3d2-6f79-4aa2-8471-a2348767724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:53.432897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1078/2172] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:53.443405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:53.444914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:53.506988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92656c8-c2e4-474c-8364-b2fbdf5d0c78 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:53.510591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedaf2d-0d12-4128-8c04-dd35d631b5df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:53.511315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedaf2d-0d12-4128-8c04-dd35d631b5df service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:53.511601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedaf2d-0d12-4128-8c04-dd35d631b5df service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:53.511910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedaf2d-0d12-4128-8c04-dd35d631b5df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:53.525975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedaf2d-0d12-4128-8c04-dd35d631b5df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:53.527477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedaf2d-0d12-4128-8c04-dd35d631b5df service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:53.539175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1095/2173] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:53.892358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf768729-4a8b-45c7-a26e-db8ea905d7aa None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:53.892767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf768729-4a8b-45c7-a26e-db8ea905d7aa None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:53.893115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf768729-4a8b-45c7-a26e-db8ea905d7aa None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:53.894832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1079/2174] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:58:53.905681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:53.906197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:53.906297 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:53.943962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:53.950492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:53.950492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:54.043807 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1096/2175] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:54.057310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf768729-4a8b-45c7-a26e-db8ea905d7aa None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:54.060099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cf768729-4a8b-45c7-a26e-db8ea905d7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:54.151784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf768729-4a8b-45c7-a26e-db8ea905d7aa None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:54.156039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1b39c-5d30-4620-89fc-9997abfadba0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:54.156907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b39c-5d30-4620-89fc-9997abfadba0 service glance] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.157513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b39c-5d30-4620-89fc-9997abfadba0 service glance] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.157513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b39c-5d30-4620-89fc-9997abfadba0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.165139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:54.169507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1b39c-5d30-4620-89fc-9997abfadba0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:54.171024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1b39c-5d30-4620-89fc-9997abfadba0 service glance] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:54.182240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:54.183915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:54.194651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1080/2176] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:58:54.221391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950623f-a71f-4de6-95c2-50c8f2cbacdd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:54.226745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:54.228997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.229568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.229679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] PATH_INFO: `/v3/projects/a2836c0ab903447597888f1b062ed0f1` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.231027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2836c0ab903447597888f1b062ed0f1)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:54.233065 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:54.361069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2836c0ab903447597888f1b062ed0f1', 'request_id': 'req-6dcddd50-4555-471f-be53-388337b7e0f2'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:54.367953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2836c0ab903447597888f1b062ed0f1', 'request_id': 'req-6dcddd50-4555-471f-be53-388337b7e0f2'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:54.421986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08066-2bd1-41fc-b70f-5a412c4e9230 None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.422692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08066-2bd1-41fc-b70f-5a412c4e9230 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.422692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08066-2bd1-41fc-b70f-5a412c4e9230 None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.424154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1081/2177] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 14:58:54.428584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.428874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.429308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.444633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2836c0ab903447597888f1b062ed0f1', 'request_id': 'req-6dcddd50-4555-471f-be53-388337b7e0f2'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:54.450043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] MFA Rules not processed for user `bead604b09be482c906a97b53bb03d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:54.454098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:54.458721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] The token cache is being invalidate because project a2836c0ab903447597888f1b062ed0f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:58:54.459218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:54.459954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6dcddd50-4555-471f-be53-388337b7e0f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:58:54.461940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1097/2178] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:58:54 2022] DELETE /identity/v3/projects/a2836c0ab903447597888f1b062ed0f1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:58:54.466541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a75f6-95fc-40c2-8173-aadddaec6e49 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.466910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a75f6-95fc-40c2-8173-aadddaec6e49 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.467281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a75f6-95fc-40c2-8173-aadddaec6e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.489975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-9e8a75f6-95fc-40c2-8173-aadddaec6e49 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:54.490521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9e8a75f6-95fc-40c2-8173-aadddaec6e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:54.509086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1082/2179] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:54.523852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:54.528708 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:54.541452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa8ee1-28b6-4477-b54e-213e11a45f84 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:54.545603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:54.546303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.546585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.546919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.562433 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:54.563898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:54.566377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1098/2180] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:54.572777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1083/2181] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:58:54.574903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7fb82-d28a-4962-bbb6-fef0724580af None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.575447 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7fb82-d28a-4962-bbb6-fef0724580af None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.576064 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7fb82-d28a-4962-bbb6-fef0724580af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.598319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-fae7fb82-d28a-4962-bbb6-fef0724580af None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:54.603921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fae7fb82-d28a-4962-bbb6-fef0724580af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:54.610366 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1099/2182] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:54.614461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:54.650212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c4093-6c86-4591-8bac-58a6dbc2cd72 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:54.662787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:54.663425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.663783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.664132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.665659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:58:54.666748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:54.668475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:54.677632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c71573-0ab2-48bd-aa0b-aef5f1d6f4fb None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.678959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c71573-0ab2-48bd-aa0b-aef5f1d6f4fb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.679439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c71573-0ab2-48bd-aa0b-aef5f1d6f4fb None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.681261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1100/2183] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 14:58:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 14:58:54.689100 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9e9b5-09e4-4623-8e2b-3703839f9068 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.689724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9e9b5-09e4-4623-8e2b-3703839f9068 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.690240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9e9b5-09e4-4623-8e2b-3703839f9068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.696889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1084/2184] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:58:54.700146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:54.717473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-cac9e9b5-09e4-4623-8e2b-3703839f9068 None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:54.720129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cac9e9b5-09e4-4623-8e2b-3703839f9068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:54.723614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584520b-8b6d-4c14-ac8a-eb85adf87300 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:54.739402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:54.740308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.740637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.740866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.741626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:54.743535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:54.746152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:54.816548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1101/2185] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:58:54.843694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9e9b5-09e4-4623-8e2b-3703839f9068 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:54.854626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9e9b5-09e4-4623-8e2b-3703839f9068 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:54.859270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1085/2186] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:58:54 2022] POST /identity/v3/projects => generated 389 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:54.864243 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:54.865652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:54.867832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.867832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.867832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.881855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:54.883695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:54.901206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1102/2187] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:58:54.955995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efe736-f55a-4321-8ff7-5ead204aa527 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:54.961463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:54.962276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:54.962648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:54.962942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:54.963763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:54.967606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:55.072442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1086/2188] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:58:54 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:58:55.077227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:55.090284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04127ba7-de47-4601-a395-9fc02ea859e9 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:55.095794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:55.096367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:55.096651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:55.096924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:55.097655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:55.099460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:55.116659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1103/2189] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:58:55.117893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:55.129267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec81f7-2c31-45c4-941b-1a1038c81597 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:55.134102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb386b7a-e181-4bad-900c-aadbbd3bef28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:55.134948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386b7a-e181-4bad-900c-aadbbd3bef28 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:55.135236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386b7a-e181-4bad-900c-aadbbd3bef28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:55.135503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386b7a-e181-4bad-900c-aadbbd3bef28 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:55.136625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386b7a-e181-4bad-900c-aadbbd3bef28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:55.138320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386b7a-e181-4bad-900c-aadbbd3bef28 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:55.150223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1087/2190] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:58:55.153943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:55.165144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb4ab-343e-4f7c-95c3-f0bed92b2af9 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:55.170527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:55.171504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:55.172698 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:55.173272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] PATH_INFO: `/v3/projects/95d896983568410c8b7208d9be8eba05/users/5ee377e9f58147bfa2f36d93a7b9e61f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:55.193420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d896983568410c8b7208d9be8eba05, user_id=5ee377e9f58147bfa2f36d93a7b9e61f, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:55.197022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:55.302104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1104/2191] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:58:55 2022] PUT /identity/v3/projects/95d896983568410c8b7208d9be8eba05/users/5ee377e9f58147bfa2f36d93a7b9e61f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 14:58:55.317268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5d622-ffe8-4bc7-83b1-417c1342ca7f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:55.317551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5d622-ffe8-4bc7-83b1-417c1342ca7f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:55.317833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5d622-ffe8-4bc7-83b1-417c1342ca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:55.349705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b7f5d622-ffe8-4bc7-83b1-417c1342ca7f None None] MFA Rules not processed for user `5ee377e9f58147bfa2f36d93a7b9e61f`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:58:55.350960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b7f5d622-ffe8-4bc7-83b1-417c1342ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:55.387640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:58:55.433725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1088/2192] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:58:55.445521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:58:55.447409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:58:55.533589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdf00c-6229-46c4-a02d-cbe9fef5b2c3 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:58:55.537377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:58:55.538031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:58:55.538365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:58:55.538644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:58:55.548189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:58:55.549645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:58:55.560303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1105/2193] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 14:59:05.555513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:05.597559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=17 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:10.117099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee75f89-9518-4b1d-b9bc-1c08ae4aa0dd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:10.117663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee75f89-9518-4b1d-b9bc-1c08ae4aa0dd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:10.117663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee75f89-9518-4b1d-b9bc-1c08ae4aa0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:10.167038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-5ee75f89-9518-4b1d-b9bc-1c08ae4aa0dd None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:10.168254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5ee75f89-9518-4b1d-b9bc-1c08ae4aa0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:10.234493 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1089/2194] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:10.259279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:10.260711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:10.362890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1be6f-7be0-4066-9f7c-4b1fc397590c service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:10.367997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5f0f9-9a9e-4c20-b6f6-8ca2d44f66fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:10.368643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5f0f9-9a9e-4c20-b6f6-8ca2d44f66fa service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:10.368926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5f0f9-9a9e-4c20-b6f6-8ca2d44f66fa service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:10.369207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5f0f9-9a9e-4c20-b6f6-8ca2d44f66fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:10.378029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5f0f9-9a9e-4c20-b6f6-8ca2d44f66fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:10.379659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5f0f9-9a9e-4c20-b6f6-8ca2d44f66fa service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:10.387627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1106/2195] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:59:20.232066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=40 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:20.384430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=41 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:21.984399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee75f89-9518-4b1d-b9bc-1c08ae4aa0dd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:22.057414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee75f89-9518-4b1d-b9bc-1c08ae4aa0dd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:22.064133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:22.065032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:22.065320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:22.065607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] PATH_INFO: `/v3/users/5ee377e9f58147bfa2f36d93a7b9e61f` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:22.066893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee377e9f58147bfa2f36d93a7b9e61f)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:22.068883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:22.417859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee377e9f58147bfa2f36d93a7b9e61f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.418837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee377e9f58147bfa2f36d93a7b9e61f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.419222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee377e9f58147bfa2f36d93a7b9e61f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.447596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee377e9f58147bfa2f36d93a7b9e61f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.576326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee377e9f58147bfa2f36d93a7b9e61f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:22.589581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1090/2196] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:21 2022] DELETE /identity/v3/users/5ee377e9f58147bfa2f36d93a7b9e61f => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:22.594300 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:22.594804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:22.594917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:22.613815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:22.615078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:22.721538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1107/2197] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:22.731859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:22.755107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:22.815908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398b77e-7f73-48e0-bff7-3967fb37f951 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:22.819627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b204f5-8a15-447c-b601-11855193a942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:22.820286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b204f5-8a15-447c-b601-11855193a942 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:22.820977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b204f5-8a15-447c-b601-11855193a942 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:22.820977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b204f5-8a15-447c-b601-11855193a942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:22.833938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b204f5-8a15-447c-b601-11855193a942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:22.838058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b204f5-8a15-447c-b601-11855193a942 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:22.847374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1091/2198] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:23.538211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:23.539983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:23.559846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55cd79-e6b3-43b2-b764-b9b49c6e0bfb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:23.564704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:23.565578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:23.565859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:23.566135 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] PATH_INFO: `/v3/projects/95d896983568410c8b7208d9be8eba05` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:23.567347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d896983568410c8b7208d9be8eba05)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:23.569465 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:24.064831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d896983568410c8b7208d9be8eba05', 'request_id': 'req-f49534e8-6c0e-427d-8918-a35b036ee775'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:24.071039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d896983568410c8b7208d9be8eba05', 'request_id': 'req-f49534e8-6c0e-427d-8918-a35b036ee775'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:24.261458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d896983568410c8b7208d9be8eba05', 'request_id': 'req-f49534e8-6c0e-427d-8918-a35b036ee775'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:24.272383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] The token cache is being invalidate because project 95d896983568410c8b7208d9be8eba05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:24.272667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:24.273283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f49534e8-6c0e-427d-8918-a35b036ee775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:24.274851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1108/2199] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:23 2022] DELETE /identity/v3/projects/95d896983568410c8b7208d9be8eba05 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:24.280683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e51fda-7d15-484e-ad0b-2f889eb0801f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.281222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e51fda-7d15-484e-ad0b-2f889eb0801f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.281222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e51fda-7d15-484e-ad0b-2f889eb0801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.299439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d6e51fda-7d15-484e-ad0b-2f889eb0801f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:24.300670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d6e51fda-7d15-484e-ad0b-2f889eb0801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:24.317983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:24.362257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1092/2200] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:24.369793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1154d9-7e46-4c0e-8804-827cc0c1174d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.370410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1154d9-7e46-4c0e-8804-827cc0c1174d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.370780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1154d9-7e46-4c0e-8804-827cc0c1174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.389655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-2c1154d9-7e46-4c0e-8804-827cc0c1174d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:24.394997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2c1154d9-7e46-4c0e-8804-827cc0c1174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:24.398601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1109/2201] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:24.403853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e51fda-7d15-484e-ad0b-2f889eb0801f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:24.413759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e51fda-7d15-484e-ad0b-2f889eb0801f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:24.418463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:24.420459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.420459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.420459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.420459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 14:59:24.420912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:24.422299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:24.433664 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1093/2202] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 14:59:24.437809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1154d9-7e46-4c0e-8804-827cc0c1174d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:24.447012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1154d9-7e46-4c0e-8804-827cc0c1174d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:24.452418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:24.453336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.453614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.453885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.454851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:24.456986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:24.573119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1110/2203] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 14:59:24 2022] POST /identity/v3/projects => generated 399 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 14:59:24.577626 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:24.587889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14081ffe-e000-4edb-bdd2-9dbe6b5565f9 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:24.656089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:24.656779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.657090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.657366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.658075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:24.660528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:24.770769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1094/2204] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 14:59:24 2022] POST /identity/v3/users => generated 300 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 14:59:24.774789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:24.785259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad247f-6d16-4540-b7bd-f3b5fbf2d7f2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:24.789980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:24.790734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.791013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.791287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.792043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:24.793665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:24.803909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1111/2205] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:59:24.808296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:24.818531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cb7dd-6ea6-4686-a779-b3bff02298c6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:24.823877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5282847-fae5-457b-82e5-1ab243aa30ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:24.824527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5282847-fae5-457b-82e5-1ab243aa30ec admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.824827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5282847-fae5-457b-82e5-1ab243aa30ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.825103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5282847-fae5-457b-82e5-1ab243aa30ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.825831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5282847-fae5-457b-82e5-1ab243aa30ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:24.827515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5282847-fae5-457b-82e5-1ab243aa30ec admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:24.837586 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1095/2206] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 14:59:24.841818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:24.851854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94316478-092e-4d46-9667-d0160a548b20 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:24.856656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:24.859146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.859146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.859146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] PATH_INFO: `/v3/projects/e9fd0ebbd9b743d0813b7614b4ab3edf/users/26474b4759474844afc54b8e09791db3/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:24.873381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fd0ebbd9b743d0813b7614b4ab3edf, user_id=26474b4759474844afc54b8e09791db3, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:24.876092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:24.971161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1112/2207] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 14:59:24 2022] PUT /identity/v3/projects/e9fd0ebbd9b743d0813b7614b4ab3edf/users/26474b4759474844afc54b8e09791db3/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:24.983341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0185-c7d6-4ef8-86ec-1c9aa36d1ac2 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:24.983636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0185-c7d6-4ef8-86ec-1c9aa36d1ac2 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:24.983915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0185-c7d6-4ef8-86ec-1c9aa36d1ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:25.010133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-bc9e0185-c7d6-4ef8-86ec-1c9aa36d1ac2 None None] MFA Rules not processed for user `26474b4759474844afc54b8e09791db3`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:25.011585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bc9e0185-c7d6-4ef8-86ec-1c9aa36d1ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:25.073701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1096/2208] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:25.084908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:25.086645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:25.110141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:25.149266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf8d9-a2c3-4dc8-99e6-0bed6dd00ea6 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:25.154937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0ff4e-76d0-40d2-bfcf-5174ebef6abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:25.155662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ff4e-76d0-40d2-bfcf-5174ebef6abd service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:25.155951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ff4e-76d0-40d2-bfcf-5174ebef6abd service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:25.156233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ff4e-76d0-40d2-bfcf-5174ebef6abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:25.166493 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0ff4e-76d0-40d2-bfcf-5174ebef6abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:25.171724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0ff4e-76d0-40d2-bfcf-5174ebef6abd service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:25.177937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1113/2209] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:34.484502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e0185-c7d6-4ef8-86ec-1c9aa36d1ac2 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:34.507351 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:34.560599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e0185-c7d6-4ef8-86ec-1c9aa36d1ac2 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:34.565598 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:34.566581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:34.566954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:34.567301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] PATH_INFO: `/v3/users/26474b4759474844afc54b8e09791db3` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:34.568663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26474b4759474844afc54b8e09791db3)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:34.570751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:34.893389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26474b4759474844afc54b8e09791db3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:34.894244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26474b4759474844afc54b8e09791db3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:34.894715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26474b4759474844afc54b8e09791db3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:34.907671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26474b4759474844afc54b8e09791db3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:35.027253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26474b4759474844afc54b8e09791db3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:35.039734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1097/2210] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 14:59:34 2022] DELETE /identity/v3/users/26474b4759474844afc54b8e09791db3 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:35.044227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:35.044743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:35.044839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:35.062017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:35.063173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:35.121119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1114/2211] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 14:59:35.130760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:35.132162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:35.188914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9859f-a2be-4ef2-aac0-d7074cbeb1e9 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:35.192490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9d549-ada4-4635-89b8-c70a6b3a22a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:35.193131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9d549-ada4-4635-89b8-c70a6b3a22a1 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:35.193414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9d549-ada4-4635-89b8-c70a6b3a22a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:35.193690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9d549-ada4-4635-89b8-c70a6b3a22a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:35.203617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9d549-ada4-4635-89b8-c70a6b3a22a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:35.208120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9d549-ada4-4635-89b8-c70a6b3a22a1 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:35.215091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1098/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:35.735877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:35.746034 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:35.751421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36db71-0441-4ca1-9488-a7b2dafa561c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:35.756696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:35.757783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:35.758211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:35.758495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] PATH_INFO: `/v3/projects/e9fd0ebbd9b743d0813b7614b4ab3edf` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:35.760880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fd0ebbd9b743d0813b7614b4ab3edf)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:35.762901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:35.999044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fd0ebbd9b743d0813b7614b4ab3edf', 'request_id': 'req-b627cd32-18c9-47a7-8865-90a52cda11d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:36.005394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fd0ebbd9b743d0813b7614b4ab3edf', 'request_id': 'req-b627cd32-18c9-47a7-8865-90a52cda11d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:36.151385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fd0ebbd9b743d0813b7614b4ab3edf', 'request_id': 'req-b627cd32-18c9-47a7-8865-90a52cda11d9'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:36.163183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] The token cache is being invalidate because project e9fd0ebbd9b743d0813b7614b4ab3edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 14:59:36.163587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:36.164234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b627cd32-18c9-47a7-8865-90a52cda11d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 14:59:36.166382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1115/2213] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 14:59:35 2022] DELETE /identity/v3/projects/e9fd0ebbd9b743d0813b7614b4ab3edf => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 14:59:44.610870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:44.611383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:44.611520 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:44.628729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] MFA Rules not processed for user `4e9c4ff3d13d4332837f519214b9df37`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:44.629917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:44.662042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:44.684827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1099/2214] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:45.738120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:45.738581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:45.738716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:45.758328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 14:59:45.760265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:45.809496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 14:59:45.823346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1116/2215] 10.222.0.23 () {58 vars in 967 bytes} [Wed Jun 15 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 14:59:45.836062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:45.837622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:45.905897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d468d-d7e0-4c4f-9f4a-f9bff056d66e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:45.910853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5ab08-6a42-4318-bbb9-4bb5ac8389d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:45.911496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5ab08-6a42-4318-bbb9-4bb5ac8389d8 service swift] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:45.911776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5ab08-6a42-4318-bbb9-4bb5ac8389d8 service swift] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:45.912049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5ab08-6a42-4318-bbb9-4bb5ac8389d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:45.920355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5ab08-6a42-4318-bbb9-4bb5ac8389d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:45.921941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5ab08-6a42-4318-bbb9-4bb5ac8389d8 service swift] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:45.930680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1100/2216] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 14:59:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 14:59:53.996037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 14:59:53.997484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:54.062763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63832a-acce-4b09-9745-6b552cf6eedc None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 14:59:54.066555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfe378-0626-4ae2-90c0-6d3587ad9d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 14:59:54.067298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfe378-0626-4ae2-90c0-6d3587ad9d0c service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 14:59:54.067645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfe378-0626-4ae2-90c0-6d3587ad9d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 14:59:54.067984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfe378-0626-4ae2-90c0-6d3587ad9d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 14:59:54.070646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b9bfe378-0626-4ae2-90c0-6d3587ad9d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 14:59:54.143181 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfe378-0626-4ae2-90c0-6d3587ad9d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 14:59:54.144863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfe378-0626-4ae2-90c0-6d3587ad9d0c service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 14:59:54.157875 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1117/2217] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 14:59:55.926540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=39 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:04.152446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=54 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:14.851612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:14.852131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:14.852131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:14.870332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] MFA Rules not processed for user `cbc9d5ce4a3241ea898f216543114fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:00:14.871549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:14.930769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1101/2218] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:00:21.131161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1151-3003-490e-b586-06e171bd3ebd None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:21.131634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1151-3003-490e-b586-06e171bd3ebd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:21.131725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1151-3003-490e-b586-06e171bd3ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:21.149490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4f9a1151-3003-490e-b586-06e171bd3ebd None None] MFA Rules not processed for user `e367e391e68a477d861a515f846a8dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:00:21.150709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4f9a1151-3003-490e-b586-06e171bd3ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:21.209573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1118/2219] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:00:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:00:24.921461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=25 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:31.201548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=25 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:37.659818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:37.661628 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:37.675344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5330b-a644-471a-acab-66b82d809fff None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:37.679535 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:37.680192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:37.680476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:37.683776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:37.683776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:37.757257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:37.758828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:37.770197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1102/2220] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:41.749689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1151-3003-490e-b586-06e171bd3ebd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:41.773274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1151-3003-490e-b586-06e171bd3ebd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:41.777189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:41.779334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:41.779699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:41.783236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:41.783236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:41.796626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:41.798312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:41.809776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1119/2221] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:47.765029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=34 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:00:49.754686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:49.756156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:49.813979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc418e-7df9-4a68-b7f8-7b52123cb765 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:49.817671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:49.818354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:49.818639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:49.818920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:49.820700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:49.868098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:49.869553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:49.882110 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:49.883588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:49.884380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f8752-9034-40f6-88c3-667af761fff0 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:49.889882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:49.890635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:49.890950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:49.891284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:49.893420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1103/2222] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:49.901796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:49.903436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:49.913626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1120/2223] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:58.099233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:00:58.105790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:58.120196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2bda-ff04-4f17-bf36-ef16f845eed2 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:00:58.127001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb1dee-b846-47a8-b01a-0521f9243f79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:00:58.127728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1dee-b846-47a8-b01a-0521f9243f79 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:00:58.128077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1dee-b846-47a8-b01a-0521f9243f79 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:00:58.128426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1dee-b846-47a8-b01a-0521f9243f79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:00:58.130426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-52cb1dee-b846-47a8-b01a-0521f9243f79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:00:58.145196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb1dee-b846-47a8-b01a-0521f9243f79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:00:58.147214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb1dee-b846-47a8-b01a-0521f9243f79 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:00:58.159784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1104/2224] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:00:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:00:59.909230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=12 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:00.043243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:00.053843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb188b42-8532-4bbc-9929-1fd462425883 service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:00.059141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:00.059828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:00.060182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:00.060659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:00.063205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:00.075476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:00.081999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:00.090079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1121/2225] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:03.945690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:03.946211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:03.946309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:03.963531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] MFA Rules not processed for user `bf1b5d8cbfbb450da8878719c4aa6712`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:03.964725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:03.971987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1105/2226] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:04.308766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:04.319675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2881-4110-4eb8-881b-7667c9f2ee1b service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:04.323464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:04.330080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:04.330080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:04.330080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:04.330080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:04.338455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:04.340635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:04.351112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1122/2227] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:04.765885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:04.767510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:04.852971 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a47ab-55de-4ff5-ab9a-eeeab69763c9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:04.857914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:04.859150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:04.859435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:04.859710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:04.861511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:04.873050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:04.875324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:04.886091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1106/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:10.929279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:10.931495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:11.007768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97ab2-9af6-40b7-80d4-a382b5276042 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:11.013019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:11.014608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:11.014935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:11.015216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] PATH_INFO: `/v3/users/4e9c4ff3d13d4332837f519214b9df37` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:11.016568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9c4ff3d13d4332837f519214b9df37)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:11.018575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:11.259915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9c4ff3d13d4332837f519214b9df37'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.260571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9c4ff3d13d4332837f519214b9df37'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.267724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9c4ff3d13d4332837f519214b9df37'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.337403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9c4ff3d13d4332837f519214b9df37'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.350950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c6eee2e3-149f-4393-adc5-7676c13355e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9c4ff3d13d4332837f519214b9df37'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:11.357014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1123/2229] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:01:10 2022] DELETE /identity/v3/users/4e9c4ff3d13d4332837f519214b9df37 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:11.629939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:11.641974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce0b7-5a9b-4eb5-acdd-83aa340c61c6 service placement] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:11.667442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:11.708023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:11.708892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:11.709173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:11.709448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] PATH_INFO: `/v3/projects/421ba5fd9c9d4ea0b3b3dd89069aabb2` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:11.710685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] RBAC: Authorizing `identity:delete_project(project_id=421ba5fd9c9d4ea0b3b3dd89069aabb2)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:11.712722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:11.904252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421ba5fd9c9d4ea0b3b3dd89069aabb2', 'request_id': 'req-c48eeb06-99f0-4f40-bd05-b26c451d395d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:12.000974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421ba5fd9c9d4ea0b3b3dd89069aabb2', 'request_id': 'req-c48eeb06-99f0-4f40-bd05-b26c451d395d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:12.007379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421ba5fd9c9d4ea0b3b3dd89069aabb2', 'request_id': 'req-c48eeb06-99f0-4f40-bd05-b26c451d395d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:12.020134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] The token cache is being invalidate because project 421ba5fd9c9d4ea0b3b3dd89069aabb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:01:12.020420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:12.021083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c48eeb06-99f0-4f40-bd05-b26c451d395d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:12.022696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1107/2230] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:01:11 2022] DELETE /identity/v3/projects/421ba5fd9c9d4ea0b3b3dd89069aabb2 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:12.027621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29727b-3ed9-4574-a06e-9ebe7972ed2c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.028091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29727b-3ed9-4574-a06e-9ebe7972ed2c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.028186 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29727b-3ed9-4574-a06e-9ebe7972ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.048396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1f29727b-3ed9-4574-a06e-9ebe7972ed2c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:12.049611 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1f29727b-3ed9-4574-a06e-9ebe7972ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:12.097497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:12.109750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1124/2231] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:12.117780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183c2c1-37fb-4913-9b28-bd533691d1fd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.118077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183c2c1-37fb-4913-9b28-bd533691d1fd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.118393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183c2c1-37fb-4913-9b28-bd533691d1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.136378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-0183c2c1-37fb-4913-9b28-bd533691d1fd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:12.137579 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0183c2c1-37fb-4913-9b28-bd533691d1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:12.146034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1108/2232] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:12.150362 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29727b-3ed9-4574-a06e-9ebe7972ed2c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:12.161521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29727b-3ed9-4574-a06e-9ebe7972ed2c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:12.166233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:12.167663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.167946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.168218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.168946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:01:12.169252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:12.170970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:12.183855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1125/2233] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:01:12.188533 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183c2c1-37fb-4913-9b28-bd533691d1fd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:12.197913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183c2c1-37fb-4913-9b28-bd533691d1fd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:12.203263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:12.203931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.204215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.204491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.205187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:12.207198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:12.323546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1109/2234] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:01:12 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:12.328607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:12.343988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a584d-3c3f-4090-a03a-1dda63a6c10d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:12.419201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:12.420098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.420521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.420932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.421983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:12.425147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:12.532672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1126/2235] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:01:12 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:12.536429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:12.547519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ea060-1ad3-4ce1-ae07-1389c1f06e75 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:12.552403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:12.553076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.553384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.553661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.554448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:12.556141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:12.567182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1110/2236] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:12.571399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:12.582419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63227231-3f4f-41cb-b20f-e8b8c60d465d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:12.587082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9b764-dabe-4ed2-b60c-773a57496644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:12.587794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9b764-dabe-4ed2-b60c-773a57496644 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.588080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9b764-dabe-4ed2-b60c-773a57496644 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.588355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9b764-dabe-4ed2-b60c-773a57496644 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.589087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9b764-dabe-4ed2-b60c-773a57496644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:12.590719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9b764-dabe-4ed2-b60c-773a57496644 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:12.601552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1127/2237] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:12.608426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:12.616470 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa923-becd-4c4f-a381-3854e6b17f05 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:12.621003 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:12.621871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.622152 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.622455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] PATH_INFO: `/v3/projects/547883446fb944e4811768d29eb67583/users/71f3bb1357f14481bd1850c1153bc253/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.638252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=547883446fb944e4811768d29eb67583, user_id=71f3bb1357f14481bd1850c1153bc253, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:12.642210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:12.738355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1111/2238] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:12 2022] PUT /identity/v3/projects/547883446fb944e4811768d29eb67583/users/71f3bb1357f14481bd1850c1153bc253/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:12.743951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.744265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.744545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.762895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:12.764655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:12.823180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1128/2239] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:12.833048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:12.834744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:12.897320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a039-fa04-4c13-9235-009adf10c3ce admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:12.902201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:12.902861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:12.903437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:12.903437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:12.911076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:12.916282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:12.916651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:12.927934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1112/2240] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:16.128221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:16.129684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:16.191959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926741e9-4d81-4d77-8512-95d8ca89b149 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:16.196116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19ffe6-e6a3-4789-9b64-b8031e9ab9a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:16.196731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ffe6-e6a3-4789-9b64-b8031e9ab9a2 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:16.197138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ffe6-e6a3-4789-9b64-b8031e9ab9a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:16.197417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ffe6-e6a3-4789-9b64-b8031e9ab9a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:16.199402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ee19ffe6-e6a3-4789-9b64-b8031e9ab9a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:16.214842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:16.268711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19ffe6-e6a3-4789-9b64-b8031e9ab9a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:16.270207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19ffe6-e6a3-4789-9b64-b8031e9ab9a2 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:16.281169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1129/2241] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:17.485406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:17.487067 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:17.579059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8faa-a4bf-4c76-92e3-93a3de30079c service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:17.584566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:17.585222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:17.585508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:17.585788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:17.588012 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:17.601900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:17.603790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:17.614997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1113/2242] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:18.766134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbaa74b-d643-4a6e-ab0b-1b85e6ccac49 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:18.767004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbaa74b-d643-4a6e-ab0b-1b85e6ccac49 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:18.767453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbaa74b-d643-4a6e-ab0b-1b85e6ccac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:18.800383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6fbaa74b-d643-4a6e-ab0b-1b85e6ccac49 None None] MFA Rules not processed for user `71f3bb1357f14481bd1850c1153bc253`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:18.801612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6fbaa74b-d643-4a6e-ab0b-1b85e6ccac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:18.871679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1130/2243] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:18.877016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:18.888286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11396d7e-e47c-4fd3-9a72-a6dde5a42651 service nova] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:18.893099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:18.893706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:18.893991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:18.894300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:18.895044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:18.896991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:19.038001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1114/2244] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:01:18 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:01:19.041591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbaa74b-d643-4a6e-ab0b-1b85e6ccac49 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:19.055424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbaa74b-d643-4a6e-ab0b-1b85e6ccac49 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:19.159943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:19.161025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:19.161562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:19.161976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:19.162914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:19.165105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:19.326057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1131/2245] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:01:19 2022] POST /identity/v3/users => generated 296 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:19.330785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:19.341770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef86ab-ee1d-45c6-ac06-93db65f821a5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:19.346790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:19.347816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:19.348177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:19.348529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:19.349351 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:19.351092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:19.363903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1115/2246] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:19.368109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:19.381036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346d31-3a68-4103-9019-5fcec14fd43a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:19.386066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:19.387143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:19.387491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:19.387770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] PATH_INFO: `/v3/projects/e050135bf2984c6188050907025aa1ac/users/1eff21757485434680444a2c5293a477/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:19.408191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e050135bf2984c6188050907025aa1ac, user_id=1eff21757485434680444a2c5293a477, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:19.411328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:19.532033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1132/2247] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:19 2022] PUT /identity/v3/projects/e050135bf2984c6188050907025aa1ac/users/1eff21757485434680444a2c5293a477/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:19.533279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:19.544574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc5c23-9e33-4a10-8b53-18b31153d04c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:19.629702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:19.630532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:19.630877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:19.631217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:19.632412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:19.633881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:19.650367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1116/2248] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:19.661937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:19.663733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f287077-220d-469a-9849-6030d8801fa3 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:19.669027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a486bc-5024-4cf7-bc75-9d8cfeb17c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:19.671112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a486bc-5024-4cf7-bc75-9d8cfeb17c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:19.671398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a486bc-5024-4cf7-bc75-9d8cfeb17c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:19.671673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a486bc-5024-4cf7-bc75-9d8cfeb17c98 admin admin] PATH_INFO: `/v3/projects/e050135bf2984c6188050907025aa1ac/users/1eff21757485434680444a2c5293a477/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:19.673720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a486bc-5024-4cf7-bc75-9d8cfeb17c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e050135bf2984c6188050907025aa1ac, user_id=1eff21757485434680444a2c5293a477, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:19.676851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a486bc-5024-4cf7-bc75-9d8cfeb17c98 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:19.680647 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:19.692781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1133/2249] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:19 2022] PUT /identity/v3/projects/e050135bf2984c6188050907025aa1ac/users/1eff21757485434680444a2c5293a477/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:21.748878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:21.752129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:21.826922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:21.841463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44b09b-1e3f-4f08-8652-f2ffd80d48db admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:21.939047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:21.941924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:21.942237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:21.942668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:21.953846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:22.073539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:22.076371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:22.092026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1117/2250] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:26.658950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db950898-4954-4e1b-b674-c8607dd0334f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:26.659425 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db950898-4954-4e1b-b674-c8607dd0334f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:26.659508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db950898-4954-4e1b-b674-c8607dd0334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:26.701438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-db950898-4954-4e1b-b674-c8607dd0334f None None] MFA Rules not processed for user `1eff21757485434680444a2c5293a477`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:26.702716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-db950898-4954-4e1b-b674-c8607dd0334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:26.805491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1134/2251] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:26.819412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:26.821445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:26.923668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41795612-51ec-4efd-a84f-1f368dd80e1f service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:26.929185 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:26.929846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:26.930132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:26.930444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:26.993259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:27.007245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:27.008961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:27.026365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1118/2252] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:28.973676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db950898-4954-4e1b-b674-c8607dd0334f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:29.014209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db950898-4954-4e1b-b674-c8607dd0334f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:29.026542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed7ef8-4d8f-4eee-81d4-80b6da3c9087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:29.027586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7ef8-4d8f-4eee-81d4-80b6da3c9087 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:29.028159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7ef8-4d8f-4eee-81d4-80b6da3c9087 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:29.028831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7ef8-4d8f-4eee-81d4-80b6da3c9087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:29.033352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-65ed7ef8-4d8f-4eee-81d4-80b6da3c9087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:29.063287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7ef8-4d8f-4eee-81d4-80b6da3c9087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:29.065303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7ef8-4d8f-4eee-81d4-80b6da3c9087 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:29.095472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1135/2253] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:30.838853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:30.865787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116f60-2787-4a9b-b175-3def422d942f service nova] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:30.874000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:30.875358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:30.875802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:30.878469 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:30.878740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:31.001444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:31.005302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:31.047026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1119/2254] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:36.275212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb056c-0756-4e59-9a3e-d856ad5178cd None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:36.275685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb056c-0756-4e59-9a3e-d856ad5178cd None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:36.275935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb056c-0756-4e59-9a3e-d856ad5178cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:36.295323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4aeb056c-0756-4e59-9a3e-d856ad5178cd None None] MFA Rules not processed for user `df400aee39a3498d8a26d55180075ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:36.296880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4aeb056c-0756-4e59-9a3e-d856ad5178cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:36.304821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1136/2255] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:36.619808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:36.621507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:36.679902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf5ed0-c43d-4061-a4f9-f22db2529e06 service nova] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:36.683633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13276b18-8874-44ef-a52e-16d8f54ea01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:36.684804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276b18-8874-44ef-a52e-16d8f54ea01b service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:36.685086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276b18-8874-44ef-a52e-16d8f54ea01b service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:36.685361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276b18-8874-44ef-a52e-16d8f54ea01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:36.687231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-13276b18-8874-44ef-a52e-16d8f54ea01b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:36.699830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13276b18-8874-44ef-a52e-16d8f54ea01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:36.701348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13276b18-8874-44ef-a52e-16d8f54ea01b service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:36.710180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1120/2256] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:46.289510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=45 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:46.706233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=59 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:48.163067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb056c-0756-4e59-9a3e-d856ad5178cd None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:48.172918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb056c-0756-4e59-9a3e-d856ad5178cd None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:48.176709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:48.177355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:48.177634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:48.177915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:48.179819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:48.244896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:48.246746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:48.267808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1137/2257] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:51.126366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:51.126879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:51.126879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:51.147823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] MFA Rules not processed for user `71f3bb1357f14481bd1850c1153bc253`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:51.149306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:51.158279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1121/2258] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:52.408932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:52.410399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:52.488127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3f051-0c54-4132-b87a-495ef1846057 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:52.493301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:52.493988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:52.507364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:52.507364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:52.507364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:52.508652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:52.510809 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:52.520733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1138/2259] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:01:52.589410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:52.591141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:52.607266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01097c-7d23-4637-9d79-079743f6d09d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:52.612006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:52.612902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:52.614957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:52.614957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] PATH_INFO: `/v3/users/cbc9d5ce4a3241ea898f216543114fdf` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:52.614957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc9d5ce4a3241ea898f216543114fdf)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:52.616721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:52.932360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc9d5ce4a3241ea898f216543114fdf'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:52.933117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc9d5ce4a3241ea898f216543114fdf'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:52.933722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc9d5ce4a3241ea898f216543114fdf'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:52.946427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc9d5ce4a3241ea898f216543114fdf'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:53.140516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3dde6402-e610-41f8-97a1-23f691f3bd98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc9d5ce4a3241ea898f216543114fdf'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:53.154130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1122/2260] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:01:52 2022] DELETE /identity/v3/users/cbc9d5ce4a3241ea898f216543114fdf => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:53.567165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:53.580070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe18f98-60ca-45d6-a1b8-45f064e1ef70 service placement] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:53.666607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:53.668780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:53.669066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:53.669374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] PATH_INFO: `/v3/projects/65a03d17e6f54c66bb4ca7571246be87` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:53.671435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a03d17e6f54c66bb4ca7571246be87)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:53.673485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:53.942755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:53.969586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a03d17e6f54c66bb4ca7571246be87', 'request_id': 'req-51792f77-621f-4b4d-8eaf-ef19c830b048'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:53.977252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a03d17e6f54c66bb4ca7571246be87', 'request_id': 'req-51792f77-621f-4b4d-8eaf-ef19c830b048'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:54.079762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a03d17e6f54c66bb4ca7571246be87', 'request_id': 'req-51792f77-621f-4b4d-8eaf-ef19c830b048'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:54.095750 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] The token cache is being invalidate because project 65a03d17e6f54c66bb4ca7571246be87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:01:54.096048 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:54.096986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-51792f77-621f-4b4d-8eaf-ef19c830b048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:01:54.098829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1139/2261] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:01:53 2022] DELETE /identity/v3/projects/65a03d17e6f54c66bb4ca7571246be87 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:01:54.104304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9001-ac64-452f-b042-015c6ad2fe3f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.105472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9001-ac64-452f-b042-015c6ad2fe3f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.105772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9001-ac64-452f-b042-015c6ad2fe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.125177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b0fd9001-ac64-452f-b042-015c6ad2fe3f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:54.126647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b0fd9001-ac64-452f-b042-015c6ad2fe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:54.192940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1123/2262] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:01:54.201524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0241001-7405-4487-aed7-77c014861938 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.201817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0241001-7405-4487-aed7-77c014861938 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.202099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0241001-7405-4487-aed7-77c014861938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.221259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c0241001-7405-4487-aed7-77c014861938 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:54.222559 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c0241001-7405-4487-aed7-77c014861938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:54.231127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1140/2263] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:54.239175 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd9001-ac64-452f-b042-015c6ad2fe3f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:54.250066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd9001-ac64-452f-b042-015c6ad2fe3f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:54.255099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:54.256089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.256376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.256658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.257403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:01:54.257721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:54.259396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:54.279127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1124/2264] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:01:54.285541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0241001-7405-4487-aed7-77c014861938 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:54.295623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0241001-7405-4487-aed7-77c014861938 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:54.300628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:54.301302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.301589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.301870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.302613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:54.304535 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:54.505217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1141/2265] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:01:54 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:54.509342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:54.520406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e81e-5a38-494e-b0a8-2b2fe9829746 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:54.575460 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:01:54.584558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:54.585239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.585525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.585860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.586675 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:54.588676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:54.884133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1125/2266] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:01:54 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:01:54.888669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:54.899739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561500-05f4-4587-8d2a-23d4e2a3a088 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:54.904870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:54.905748 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.906040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.906707 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.907457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:54.909104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:54.920283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1142/2267] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:54.924298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:54.935661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e358a-5648-4a5a-8558-5bb5ab743d9a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:54.940281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da10d02-7179-4edb-9d88-f06e4fd14524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:54.940944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da10d02-7179-4edb-9d88-f06e4fd14524 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.941232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da10d02-7179-4edb-9d88-f06e4fd14524 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.941511 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da10d02-7179-4edb-9d88-f06e4fd14524 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.942282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da10d02-7179-4edb-9d88-f06e4fd14524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:54.944005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da10d02-7179-4edb-9d88-f06e4fd14524 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:54.957979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1126/2268] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:01:54.959240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:54.969172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95495d70-1015-414e-b7df-94d816a1c62d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:54.978139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:54.979730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:54.980026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:54.980305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] PATH_INFO: `/v3/projects/35b27a4dffff4144b63aab9d758b5687/users/f0ce9a86447a454887631cdc67deb641/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:54.996775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b27a4dffff4144b63aab9d758b5687, user_id=f0ce9a86447a454887631cdc67deb641, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:55.003240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:55.226606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1143/2269] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:01:54 2022] PUT /identity/v3/projects/35b27a4dffff4144b63aab9d758b5687/users/f0ce9a86447a454887631cdc67deb641/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:01:55.237329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ed7f-17f7-4eb6-830d-2b2663ffb9db None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:55.237926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ed7f-17f7-4eb6-830d-2b2663ffb9db None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:55.238158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ed7f-17f7-4eb6-830d-2b2663ffb9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:55.264571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6221ed7f-17f7-4eb6-830d-2b2663ffb9db None None] MFA Rules not processed for user `f0ce9a86447a454887631cdc67deb641`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:01:55.265785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6221ed7f-17f7-4eb6-830d-2b2663ffb9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:55.334828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1127/2270] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:01:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:01:55.346813 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:01:55.348531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:01:55.425748 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90096a76-e275-47a9-a5b9-6989b91bfeb6 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:01:55.432258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:01:55.433156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:01:55.433439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:01:55.433715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:01:55.441685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:01:55.443319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:01:55.452818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1144/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:01:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:01.207001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b98ed-9673-4e58-a701-bf632cb74c30 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:01.207571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b98ed-9673-4e58-a701-bf632cb74c30 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:01.207571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b98ed-9673-4e58-a701-bf632cb74c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:01.225599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6f1b98ed-9673-4e58-a701-bf632cb74c30 None None] MFA Rules not processed for user `e367e391e68a477d861a515f846a8dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:01.226857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6f1b98ed-9673-4e58-a701-bf632cb74c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:01.280975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1128/2272] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:05.450094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=6,keystone:SELECT=79,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:05.540688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:05.617904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d177f5b-ccad-4819-a2dd-cc089fe0e2b6 service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:05.624722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:05.625820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:05.626113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:05.626472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] PATH_INFO: `/v3/users/f0ce9a86447a454887631cdc67deb641` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:05.627766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0ce9a86447a454887631cdc67deb641)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:05.629751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:05.835096 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0ce9a86447a454887631cdc67deb641'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:05.835800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0ce9a86447a454887631cdc67deb641'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:05.844241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0ce9a86447a454887631cdc67deb641'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:05.974850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0ce9a86447a454887631cdc67deb641'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:06.001944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0ce9a86447a454887631cdc67deb641'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:06.016655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1145/2273] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:05 2022] DELETE /identity/v3/users/f0ce9a86447a454887631cdc67deb641 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:06.021224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:06.021731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:06.021864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:06.042631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:06.043884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:06.115794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1129/2274] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:06.126727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:06.128133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:06.189776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f914-7e54-4384-aa89-fd3cf8918f1e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:06.194080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c8997-e4c8-4770-8ff9-ab24bb4c153a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:06.194823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c8997-e4c8-4770-8ff9-ab24bb4c153a service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:06.195109 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c8997-e4c8-4770-8ff9-ab24bb4c153a service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:06.195383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c8997-e4c8-4770-8ff9-ab24bb4c153a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:06.206202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c8997-e4c8-4770-8ff9-ab24bb4c153a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:06.209513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c8997-e4c8-4770-8ff9-ab24bb4c153a service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:06.220692 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1146/2275] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:02:06.801845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:06.804687 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:06.849932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981eda7b-e696-4ba6-aaae-fe48e6968f37 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:06.859112 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:06.864197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:06.864197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:06.864197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] PATH_INFO: `/v3/projects/35b27a4dffff4144b63aab9d758b5687` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:06.864197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b27a4dffff4144b63aab9d758b5687)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:06.864814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:06.866570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:07.015795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b27a4dffff4144b63aab9d758b5687', 'request_id': 'req-039a8f32-a1b8-4a1a-b11a-f462d8984529'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:07.091464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b27a4dffff4144b63aab9d758b5687', 'request_id': 'req-039a8f32-a1b8-4a1a-b11a-f462d8984529'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:07.106789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b27a4dffff4144b63aab9d758b5687', 'request_id': 'req-039a8f32-a1b8-4a1a-b11a-f462d8984529'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:07.141581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] The token cache is being invalidate because project 35b27a4dffff4144b63aab9d758b5687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:07.143991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:07.145926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-039a8f32-a1b8-4a1a-b11a-f462d8984529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:07.150954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1130/2276] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:06 2022] DELETE /identity/v3/projects/35b27a4dffff4144b63aab9d758b5687 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:07.155033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a26ff-21db-47d0-84fb-d1aceef4169f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.155635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a26ff-21db-47d0-84fb-d1aceef4169f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.155635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a26ff-21db-47d0-84fb-d1aceef4169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.182360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-726a26ff-21db-47d0-84fb-d1aceef4169f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:07.183607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-726a26ff-21db-47d0-84fb-d1aceef4169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:07.210494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:07.271814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1147/2277] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:07.280888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409be5e8-1579-475f-9d38-31701bec4e67 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.283268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409be5e8-1579-475f-9d38-31701bec4e67 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.283268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409be5e8-1579-475f-9d38-31701bec4e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.302360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-409be5e8-1579-475f-9d38-31701bec4e67 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:07.303719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-409be5e8-1579-475f-9d38-31701bec4e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:07.313049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1131/2278] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:07.317334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a26ff-21db-47d0-84fb-d1aceef4169f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:07.328636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a26ff-21db-47d0-84fb-d1aceef4169f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:07.333838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:07.339208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.339886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.340179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.341067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:07.341508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:07.346497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:07.360876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1148/2279] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:02:07.369273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409be5e8-1579-475f-9d38-31701bec4e67 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:07.375984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409be5e8-1579-475f-9d38-31701bec4e67 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:07.380901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:07.382223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.382584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.382928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.383750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:07.387333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:07.507816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1132/2280] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:07 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:07.517859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:07.528338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9151d9-bb65-4e44-8d48-8bbf912224c2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:07.590778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:07.591473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.591789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.598284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.598284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:07.598284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:07.736051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1149/2281] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:07 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:07.739933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:07.751226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173a206-c8ca-4d35-8398-35b21099c99a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:07.755901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:07.756563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.756854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.757131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.757860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:07.759555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:07.770359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1133/2282] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:07.781372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:07.792725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018b8cf-1ae9-46a6-8205-fe4d655a1c2e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:07.797328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46f838-0734-4626-84a0-126644f741a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:07.797994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46f838-0734-4626-84a0-126644f741a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.798342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46f838-0734-4626-84a0-126644f741a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.798621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46f838-0734-4626-84a0-126644f741a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.799350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46f838-0734-4626-84a0-126644f741a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:07.801165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46f838-0734-4626-84a0-126644f741a1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:07.816708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1150/2283] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:07.818279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:07.829559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c70de4-f03b-42db-9604-4356f3b5e883 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:07.834266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:07.835166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.835537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.835761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] PATH_INFO: `/v3/projects/075d58aaf00a463aa8abef3086c7ef54/users/9668d7c47ce048c3b19c57bf4bea2be4/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:07.854928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=075d58aaf00a463aa8abef3086c7ef54, user_id=9668d7c47ce048c3b19c57bf4bea2be4, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:07.857573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:07.963636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1134/2284] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:07 2022] PUT /identity/v3/projects/075d58aaf00a463aa8abef3086c7ef54/users/9668d7c47ce048c3b19c57bf4bea2be4/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:07.974527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a2521-b89e-408f-8c65-60941fcae93f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:07.974821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a2521-b89e-408f-8c65-60941fcae93f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:07.975087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a2521-b89e-408f-8c65-60941fcae93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:08.002788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-414a2521-b89e-408f-8c65-60941fcae93f None None] MFA Rules not processed for user `9668d7c47ce048c3b19c57bf4bea2be4`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:08.003935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-414a2521-b89e-408f-8c65-60941fcae93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:08.072119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1151/2285] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:08.083657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:08.085393 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:08.163090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cf941-5163-4071-9447-1ff12ed18e07 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:08.169650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a43930-9926-4366-977c-d9cfa0ced03e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:08.170339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a43930-9926-4366-977c-d9cfa0ced03e service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:08.170625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a43930-9926-4366-977c-d9cfa0ced03e service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:08.170906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a43930-9926-4366-977c-d9cfa0ced03e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:08.178921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a43930-9926-4366-977c-d9cfa0ced03e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:08.180577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a43930-9926-4366-977c-d9cfa0ced03e service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:08.189446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1135/2286] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:11.485706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a2521-b89e-408f-8c65-60941fcae93f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:11.503346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:11.559193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a2521-b89e-408f-8c65-60941fcae93f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:11.564677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:11.565999 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:11.566461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:11.566873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] PATH_INFO: `/v3/users/9668d7c47ce048c3b19c57bf4bea2be4` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:11.568636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9668d7c47ce048c3b19c57bf4bea2be4)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:11.571094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:11.805393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9668d7c47ce048c3b19c57bf4bea2be4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:11.807204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9668d7c47ce048c3b19c57bf4bea2be4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:11.814110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9668d7c47ce048c3b19c57bf4bea2be4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:11.888095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9668d7c47ce048c3b19c57bf4bea2be4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:11.901905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9668d7c47ce048c3b19c57bf4bea2be4'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:11.908155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1152/2287] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:11 2022] DELETE /identity/v3/users/9668d7c47ce048c3b19c57bf4bea2be4 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:11.912683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:11.913166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:11.913419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:11.931930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:11.933304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:11.948791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:12.025816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1136/2288] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:12.036764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:12.038220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:12.128844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cbd1d-99ea-469a-8d3a-8a14b77d1bf7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:12.145116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1440c36-9bbc-4d9c-80b2-45b33900cb04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:12.145116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1440c36-9bbc-4d9c-80b2-45b33900cb04 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:12.145116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1440c36-9bbc-4d9c-80b2-45b33900cb04 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:12.145116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1440c36-9bbc-4d9c-80b2-45b33900cb04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:12.158781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1440c36-9bbc-4d9c-80b2-45b33900cb04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:12.160601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1440c36-9bbc-4d9c-80b2-45b33900cb04 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:12.173921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1153/2289] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:12.680076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:12.682630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:12.699289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf3230-f563-4619-8a32-1f76ba272e8d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:12.705235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:12.706250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:12.706777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:12.707150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] PATH_INFO: `/v3/projects/075d58aaf00a463aa8abef3086c7ef54` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:12.709041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=075d58aaf00a463aa8abef3086c7ef54)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:12.711863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:12.872788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '075d58aaf00a463aa8abef3086c7ef54', 'request_id': 'req-6d803693-aede-4018-86b6-8fdb0c2ff0bc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:12.990553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '075d58aaf00a463aa8abef3086c7ef54', 'request_id': 'req-6d803693-aede-4018-86b6-8fdb0c2ff0bc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:12.997328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '075d58aaf00a463aa8abef3086c7ef54', 'request_id': 'req-6d803693-aede-4018-86b6-8fdb0c2ff0bc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:13.010064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] The token cache is being invalidate because project 075d58aaf00a463aa8abef3086c7ef54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:13.010064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:13.010639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-6d803693-aede-4018-86b6-8fdb0c2ff0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:13.012451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1137/2290] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:12 2022] DELETE /identity/v3/projects/075d58aaf00a463aa8abef3086c7ef54 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:13.017333 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142bf24-e8c3-4ffe-9ba4-7c04e41d2ae6 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.017757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142bf24-e8c3-4ffe-9ba4-7c04e41d2ae6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.017921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142bf24-e8c3-4ffe-9ba4-7c04e41d2ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.037981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-0142bf24-e8c3-4ffe-9ba4-7c04e41d2ae6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:13.039240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0142bf24-e8c3-4ffe-9ba4-7c04e41d2ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:13.074703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:13.100001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1154/2291] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:13.107973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23008ee-b493-4c59-b11b-5cc22de5f89d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.108266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23008ee-b493-4c59-b11b-5cc22de5f89d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.108549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23008ee-b493-4c59-b11b-5cc22de5f89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.128899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c23008ee-b493-4c59-b11b-5cc22de5f89d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:13.130041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c23008ee-b493-4c59-b11b-5cc22de5f89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:13.138794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1138/2292] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:13.143012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142bf24-e8c3-4ffe-9ba4-7c04e41d2ae6 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:13.152837 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142bf24-e8c3-4ffe-9ba4-7c04e41d2ae6 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:13.157587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:13.158265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.158552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.158831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.159560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:13.159873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:13.161583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:13.173327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1155/2293] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:13.177684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23008ee-b493-4c59-b11b-5cc22de5f89d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:13.187913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23008ee-b493-4c59-b11b-5cc22de5f89d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:13.192585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:13.202403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.202403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.202403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.202403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:13.202403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:13.347387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1139/2294] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:13 2022] POST /identity/v3/projects => generated 403 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:13.351724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:13.362031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce4583-feff-4855-8945-ce7d2d32716d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:13.420253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:13.420854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.421141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.421423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.422114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:13.427269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:13.546315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1156/2295] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:13 2022] POST /identity/v3/users => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:13.551062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:13.564456 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db9e4-e21c-40a9-a2e8-3ca9d68f4f94 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:13.568962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:13.569620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.569904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.570296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.571025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:13.572697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:13.583317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1140/2296] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:13.588037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:13.598547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e04a-767c-4e66-9759-13c5f868a097 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:13.603206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54c86a-b1d8-44b7-9761-137d0d2caf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:13.604585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c86a-b1d8-44b7-9761-137d0d2caf71 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.605021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c86a-b1d8-44b7-9761-137d0d2caf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.605305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c86a-b1d8-44b7-9761-137d0d2caf71 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.606041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54c86a-b1d8-44b7-9761-137d0d2caf71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:13.607782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54c86a-b1d8-44b7-9761-137d0d2caf71 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:13.618576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1157/2297] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:13.623453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:13.637032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9aea-63e7-4da9-891a-12c7e1da63c6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:13.643061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:13.643961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.644245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.644561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] PATH_INFO: `/v3/projects/ac1a67678eca46988fa8c543776ab08f/users/4e5a86a9fa7647f6924e60dcb6c31d53/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.661727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac1a67678eca46988fa8c543776ab08f, user_id=4e5a86a9fa7647f6924e60dcb6c31d53, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:13.665165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:13.828931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1141/2298] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:13 2022] PUT /identity/v3/projects/ac1a67678eca46988fa8c543776ab08f/users/4e5a86a9fa7647f6924e60dcb6c31d53/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:13.848629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b9b4-5d9d-49bc-a72b-3af2b47f5bd0 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:13.849016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b9b4-5d9d-49bc-a72b-3af2b47f5bd0 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:13.849682 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16b9b4-5d9d-49bc-a72b-3af2b47f5bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:13.878586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ac16b9b4-5d9d-49bc-a72b-3af2b47f5bd0 None None] MFA Rules not processed for user `4e5a86a9fa7647f6924e60dcb6c31d53`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:13.879871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ac16b9b4-5d9d-49bc-a72b-3af2b47f5bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:13.954954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1158/2299] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:13.956164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:13.999619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:14.031361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1da9c-b66f-4848-a0e7-0600a593a94f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:14.036311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:14.037008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.037299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.037572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.038306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:14.040296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:14.132382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1142/2300] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:13 2022] POST /identity/v3/projects => generated 403 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:14.133890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b9b4-5d9d-49bc-a72b-3af2b47f5bd0 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:14.146341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16b9b4-5d9d-49bc-a72b-3af2b47f5bd0 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:14.212437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:14.213167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.213453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.213833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.214456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:14.216914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:14.319625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1159/2301] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:14 2022] POST /identity/v3/users => generated 308 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:14.320835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:14.340444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07ab21-30aa-4fb9-b7cb-10125e53781f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:14.345335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:14.346308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.346596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.346875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.347614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:14.349266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:14.360389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1143/2302] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:14.365663 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:14.376987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36561844-2b9c-4df9-b703-6afbfbd8074f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:14.383295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:14.384665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.386319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.386778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] PATH_INFO: `/v3/projects/194ba75b56d340f69b0e37d14272140b/users/95d2d719571b41a0a77aaad5271b1e9e/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.405174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] RBAC: Authorizing `identity:create_grant(project_id=194ba75b56d340f69b0e37d14272140b, user_id=95d2d719571b41a0a77aaad5271b1e9e, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:14.407918 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:14.449495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:14.500315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1160/2303] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:14 2022] PUT /identity/v3/projects/194ba75b56d340f69b0e37d14272140b/users/95d2d719571b41a0a77aaad5271b1e9e/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:14.504466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:14.515097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdee2-c508-47a1-aa39-40aed670e8e8 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:14.602024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41f9b5-6b91-44fc-9ea8-02b74fece0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:14.602905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f9b5-6b91-44fc-9ea8-02b74fece0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.603248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f9b5-6b91-44fc-9ea8-02b74fece0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.603562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f9b5-6b91-44fc-9ea8-02b74fece0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.604992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41f9b5-6b91-44fc-9ea8-02b74fece0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:14.606690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41f9b5-6b91-44fc-9ea8-02b74fece0b7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:14.617006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1144/2304] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:14.622460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:14.632628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c55c80-c9dc-4ebc-b1d2-c655d459261f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:14.638421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:14.639298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.639581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.639954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] PATH_INFO: `/v3/projects/194ba75b56d340f69b0e37d14272140b/users/95d2d719571b41a0a77aaad5271b1e9e/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.642832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=194ba75b56d340f69b0e37d14272140b, user_id=95d2d719571b41a0a77aaad5271b1e9e, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:14.645507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:14.656981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1161/2305] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:14 2022] PUT /identity/v3/projects/194ba75b56d340f69b0e37d14272140b/users/95d2d719571b41a0a77aaad5271b1e9e/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:14.667623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ccc3a-cb56-40f1-bcee-c42bb8543d80 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.667926 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ccc3a-cb56-40f1-bcee-c42bb8543d80 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.668188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ccc3a-cb56-40f1-bcee-c42bb8543d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.696131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-dd9ccc3a-cb56-40f1-bcee-c42bb8543d80 None None] MFA Rules not processed for user `95d2d719571b41a0a77aaad5271b1e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:14.698874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dd9ccc3a-cb56-40f1-bcee-c42bb8543d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:14.779538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1145/2306] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:14 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:14.790375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:14.792111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:14.857802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385fb5e-58f4-44f0-9683-85ebe1227c3b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:14.861450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485e33b-0cd0-402c-a972-d5ff9002c751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:14.862950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485e33b-0cd0-402c-a972-d5ff9002c751 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:14.862950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485e33b-0cd0-402c-a972-d5ff9002c751 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:14.862950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485e33b-0cd0-402c-a972-d5ff9002c751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:14.873236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485e33b-0cd0-402c-a972-d5ff9002c751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:14.874789 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485e33b-0cd0-402c-a972-d5ff9002c751 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:14.884360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1162/2307] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:14 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:15.044800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ccc3a-cb56-40f1-bcee-c42bb8543d80 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:15.076875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:15.110831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ccc3a-cb56-40f1-bcee-c42bb8543d80 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:15.115921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:15.116588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:15.117106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:15.117823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] PATH_INFO: `/v3/users/4e5a86a9fa7647f6924e60dcb6c31d53` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:15.120475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5a86a9fa7647f6924e60dcb6c31d53)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:15.121783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:15.331606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5a86a9fa7647f6924e60dcb6c31d53'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:15.332321 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5a86a9fa7647f6924e60dcb6c31d53'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:15.332918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5a86a9fa7647f6924e60dcb6c31d53'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:15.347002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5a86a9fa7647f6924e60dcb6c31d53'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:15.425286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5a86a9fa7647f6924e60dcb6c31d53'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:15.438770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1146/2308] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:15 2022] DELETE /identity/v3/users/4e5a86a9fa7647f6924e60dcb6c31d53 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:15.443373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:15.443864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:15.443971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:15.462445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:15.464351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:15.523289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1163/2309] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:15.533984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:15.536547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:15.601343 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9a50a-eba0-45ba-9464-758b5810bc43 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:15.606097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:15.606802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:15.607090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:15.607372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:15.617600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:15.619136 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:15.629365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1147/2310] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:16.073877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:16.076030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:16.102951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d83d7-5d0b-4dc7-b3eb-2aea9be8aec6 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:16.112439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:16.112439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:16.112439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:16.112439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] PATH_INFO: `/v3/projects/ac1a67678eca46988fa8c543776ab08f` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:16.112819 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac1a67678eca46988fa8c543776ab08f)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:16.114257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:16.204246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:16.220685 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac1a67678eca46988fa8c543776ab08f', 'request_id': 'req-0cccc30e-efd9-4a14-8f71-0c6677732889'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.228946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac1a67678eca46988fa8c543776ab08f', 'request_id': 'req-0cccc30e-efd9-4a14-8f71-0c6677732889'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.311355 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac1a67678eca46988fa8c543776ab08f', 'request_id': 'req-0cccc30e-efd9-4a14-8f71-0c6677732889'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.326928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] The token cache is being invalidate because project ac1a67678eca46988fa8c543776ab08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:16.327282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.328512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.329892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1164/2311] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:16 2022] DELETE /identity/v3/projects/ac1a67678eca46988fa8c543776ab08f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:16.333673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:16.343348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9601430-0eb4-49fe-aea2-7a98ecb7f74d service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:16.390319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:16.402289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:16.403260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:16.404059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:16.404059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] PATH_INFO: `/v3/users/95d2d719571b41a0a77aaad5271b1e9e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:16.405005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d2d719571b41a0a77aaad5271b1e9e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:16.407042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:16.617889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d2d719571b41a0a77aaad5271b1e9e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.618912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d2d719571b41a0a77aaad5271b1e9e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.620720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d2d719571b41a0a77aaad5271b1e9e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.639952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d2d719571b41a0a77aaad5271b1e9e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.751075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-049a1a63-4e6e-4711-b535-6451080a130c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d2d719571b41a0a77aaad5271b1e9e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:16.765983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1148/2312] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:16 2022] DELETE /identity/v3/users/95d2d719571b41a0a77aaad5271b1e9e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:17.233168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:17.234731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:17.249518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccc30e-efd9-4a14-8f71-0c6677732889 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:17.326141 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:17.327046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:17.327332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:17.327757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] PATH_INFO: `/v3/projects/194ba75b56d340f69b0e37d14272140b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:17.329014 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=194ba75b56d340f69b0e37d14272140b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:17.331002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:17.469213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194ba75b56d340f69b0e37d14272140b', 'request_id': 'req-38168bf9-d373-4ab8-b973-d199f45bc9d8'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:17.477026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194ba75b56d340f69b0e37d14272140b', 'request_id': 'req-38168bf9-d373-4ab8-b973-d199f45bc9d8'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:17.579699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194ba75b56d340f69b0e37d14272140b', 'request_id': 'req-38168bf9-d373-4ab8-b973-d199f45bc9d8'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:17.599489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] The token cache is being invalidate because project 194ba75b56d340f69b0e37d14272140b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:17.599905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:17.600785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-38168bf9-d373-4ab8-b973-d199f45bc9d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:17.604171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1165/2313] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:17 2022] DELETE /identity/v3/projects/194ba75b56d340f69b0e37d14272140b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:17.609109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e7e61-c743-4a3d-a754-0fb4ffdff2ab None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:17.609637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e7e61-c743-4a3d-a754-0fb4ffdff2ab None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:17.609637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e7e61-c743-4a3d-a754-0fb4ffdff2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:17.628503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-db1e7e61-c743-4a3d-a754-0fb4ffdff2ab None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:17.630001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-db1e7e61-c743-4a3d-a754-0fb4ffdff2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:17.699316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1149/2314] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:17.707138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17b515-073f-48fb-9238-edcaeeeca399 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:17.707527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17b515-073f-48fb-9238-edcaeeeca399 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:17.707895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17b515-073f-48fb-9238-edcaeeeca399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:17.726996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4b17b515-073f-48fb-9238-edcaeeeca399 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:17.735471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4b17b515-073f-48fb-9238-edcaeeeca399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:17.736361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1166/2315] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:17.740599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e7e61-c743-4a3d-a754-0fb4ffdff2ab None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:17.751038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e7e61-c743-4a3d-a754-0fb4ffdff2ab None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:17.755961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:17.756725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:17.757116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:17.757482 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:17.758350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:17.758757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:17.761599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:17.773038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1150/2316] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:02:17.784147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17b515-073f-48fb-9238-edcaeeeca399 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:17.793735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17b515-073f-48fb-9238-edcaeeeca399 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:17.798605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:17.799259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:17.799540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:17.799817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:17.800518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:17.802445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:17.999061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1167/2317] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:17 2022] POST /identity/v3/projects => generated 365 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:18.003844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.017988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de56a50-33f2-4d98-806f-2c0acdb00680 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.079484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:18.086358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.089048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.089048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.089048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.089048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.090356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.186415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1151/2318] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:18 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:18.190429 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.203782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c32c7-8c03-461a-a9fb-e544d551b62a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.208638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.209385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.218479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.218479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.218479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.218479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.224627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1168/2319] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:02:18.229706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.239939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7c895-fa18-462d-ba40-f7c5cfc26135 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.246420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.247121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.247519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.247673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.248411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.250026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.261312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1152/2320] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:18.265544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.272372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:18.278150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb6c3-e85f-4e59-9cfe-4a4e3bf83913 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.284348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb1373-ee66-4d4c-a506-99d4d579a198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.285542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1373-ee66-4d4c-a506-99d4d579a198 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.285993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1373-ee66-4d4c-a506-99d4d579a198 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.288445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1373-ee66-4d4c-a506-99d4d579a198 admin admin] PATH_INFO: `/v3/projects/57711d8976ba452eb4d2a8485b7365fc/users/6c9538cf0a834441a791527f90ab9699/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.305569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb1373-ee66-4d4c-a506-99d4d579a198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57711d8976ba452eb4d2a8485b7365fc, user_id=6c9538cf0a834441a791527f90ab9699, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.310522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb1373-ee66-4d4c-a506-99d4d579a198 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.413180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.414986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:18.433133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1169/2321] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:18 2022] PUT /identity/v3/projects/57711d8976ba452eb4d2a8485b7365fc/users/6c9538cf0a834441a791527f90ab9699/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:18.444837 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e560-d3c1-4c15-a9f1-648620f0173d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.445136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e560-d3c1-4c15-a9f1-648620f0173d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.445463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e560-d3c1-4c15-a9f1-648620f0173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.472464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-41b7e560-d3c1-4c15-a9f1-648620f0173d None None] MFA Rules not processed for user `6c9538cf0a834441a791527f90ab9699`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:18.473733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-41b7e560-d3c1-4c15-a9f1-648620f0173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:18.474954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168b0ab-24bd-4dbc-b422-7db2eee4deea admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.525725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.531162 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.531746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.533235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.535232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:18.543654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1170/2322] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:18.548361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7e560-d3c1-4c15-a9f1-648620f0173d None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.606576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.608630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.619149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1153/2323] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:02:18.619906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7e560-d3c1-4c15-a9f1-648620f0173d None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.624788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.625699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.625985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.626290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.627009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.629158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:18.740817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1171/2324] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:18 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:18.750379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:18.756139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ec9d3-ac5e-4095-91e0-bb84b269e2d9 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:18.796549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:18.814454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:18.815118 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:18.815396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:18.815672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:18.816346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:18.818573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.070597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1154/2325] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:18 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:19.070790 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.080539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb91e-669b-4173-9483-470f520a530e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.085082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.085746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.086033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.086347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.087081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.088714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.100473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1172/2326] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:02:19.104629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.114735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5143aa-93e5-435c-8269-1f009dbae1c2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.120035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.120922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.121248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.121604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] PATH_INFO: `/v3/projects/6a47e6cb906f45c38d04828a1c6ddddc/users/2f5f9bdf303e4a5482944cb674f80ed6/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.137156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a47e6cb906f45c38d04828a1c6ddddc, user_id=2f5f9bdf303e4a5482944cb674f80ed6, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.139844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.265823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1155/2327] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:19 2022] PUT /identity/v3/projects/6a47e6cb906f45c38d04828a1c6ddddc/users/2f5f9bdf303e4a5482944cb674f80ed6/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:19.269755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.279853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad51701-102f-4a18-bfdf-9d24715e6300 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.330196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:19.340704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4d4b5-0153-4fe7-afeb-e5798790316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.341378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d4b5-0153-4fe7-afeb-e5798790316d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.341664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d4b5-0153-4fe7-afeb-e5798790316d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.341992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4d4b5-0153-4fe7-afeb-e5798790316d admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.342767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d4b5-0153-4fe7-afeb-e5798790316d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.344412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4d4b5-0153-4fe7-afeb-e5798790316d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.354905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1173/2328] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:19.359093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.369997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b77a-f33e-4387-a545-c05a6094de10 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.374679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.375557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.375844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.376124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] PATH_INFO: `/v3/projects/6a47e6cb906f45c38d04828a1c6ddddc/users/2f5f9bdf303e4a5482944cb674f80ed6/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.378218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a47e6cb906f45c38d04828a1c6ddddc, user_id=2f5f9bdf303e4a5482944cb674f80ed6, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.380916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.392070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1156/2329] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:19 2022] PUT /identity/v3/projects/6a47e6cb906f45c38d04828a1c6ddddc/users/2f5f9bdf303e4a5482944cb674f80ed6/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:19.402127 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbda80-594b-444d-8941-db834ec6c513 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.402457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbda80-594b-444d-8941-db834ec6c513 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.402740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbda80-594b-444d-8941-db834ec6c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.429239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ebcbda80-594b-444d-8941-db834ec6c513 None None] MFA Rules not processed for user `2f5f9bdf303e4a5482944cb674f80ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:19.431922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ebcbda80-594b-444d-8941-db834ec6c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:19.500537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1174/2330] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:19.514387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.516089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:19.578008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1623777-7d72-4977-82b9-fac8fd4eb92e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.582089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.582792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.583080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.583358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.593195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.594781 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.605685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1157/2331] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:19.891198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbda80-594b-444d-8941-db834ec6c513 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:19.901786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbda80-594b-444d-8941-db834ec6c513 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:19.913321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6a292-6653-498e-8f92-f3fb9a983ded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:19.913321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6a292-6653-498e-8f92-f3fb9a983ded service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:19.913321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6a292-6653-498e-8f92-f3fb9a983ded service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:19.913321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6a292-6653-498e-8f92-f3fb9a983ded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:19.967928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6a292-6653-498e-8f92-f3fb9a983ded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:19.969571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6a292-6653-498e-8f92-f3fb9a983ded service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:19.982368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1175/2332] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:23.748530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:23.812104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2385a9-13f4-41b9-829e-bc19a67800b4 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:23.816658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:23.817521 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:23.817805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:23.818080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] PATH_INFO: `/v3/users/6c9538cf0a834441a791527f90ab9699` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:23.819304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c9538cf0a834441a791527f90ab9699)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:23.821245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:24.042819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:24.050950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c9538cf0a834441a791527f90ab9699'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:24.051674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c9538cf0a834441a791527f90ab9699'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:24.052271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c9538cf0a834441a791527f90ab9699'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:24.064731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c9538cf0a834441a791527f90ab9699'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:24.138819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c9538cf0a834441a791527f90ab9699'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:24.151599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1158/2333] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:23 2022] DELETE /identity/v3/users/6c9538cf0a834441a791527f90ab9699 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:24.156150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:24.156628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:24.156727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:24.174822 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:24.175952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:24.235130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1176/2334] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:24.245151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:24.246595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:24.303452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c25cb-283d-45ee-890a-3b0609246170 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:24.307631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:24.308268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:24.308549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:24.308879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:24.317934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:24.319412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:24.329305 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1159/2335] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:24.890215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:24.891756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:24.902981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:24.907126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4edd21-c4ac-4ce1-9d45-c6ad96c34ff8 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:24.913348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:24.914237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:24.914414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:24.914793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] PATH_INFO: `/v3/projects/57711d8976ba452eb4d2a8485b7365fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:24.916660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57711d8976ba452eb4d2a8485b7365fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:24.918641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:25.054024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57711d8976ba452eb4d2a8485b7365fc', 'request_id': 'req-3266d64d-55fb-4229-a4e0-6a7216a00a34'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.061482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57711d8976ba452eb4d2a8485b7365fc', 'request_id': 'req-3266d64d-55fb-4229-a4e0-6a7216a00a34'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.219471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57711d8976ba452eb4d2a8485b7365fc', 'request_id': 'req-3266d64d-55fb-4229-a4e0-6a7216a00a34'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.231176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] The token cache is being invalidate because project 57711d8976ba452eb4d2a8485b7365fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:25.231464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.235373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.235373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1177/2336] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:24 2022] DELETE /identity/v3/projects/57711d8976ba452eb4d2a8485b7365fc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:25.237878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:25.248150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de80564-be47-4bbe-a421-8ac9452f420d service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:25.305400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:25.306893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:25.307176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:25.307452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] PATH_INFO: `/v3/users/2f5f9bdf303e4a5482944cb674f80ed6` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:25.308645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f5f9bdf303e4a5482944cb674f80ed6)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:25.310632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:25.565021 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f5f9bdf303e4a5482944cb674f80ed6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.565673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f5f9bdf303e4a5482944cb674f80ed6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.566313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f5f9bdf303e4a5482944cb674f80ed6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.574481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:25.578919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f5f9bdf303e4a5482944cb674f80ed6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.761819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-53461db0-633b-48c1-8032-3b749dae179c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f5f9bdf303e4a5482944cb674f80ed6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:25.774746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1160/2337] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:25 2022] DELETE /identity/v3/users/2f5f9bdf303e4a5482944cb674f80ed6 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:26.280584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:26.282079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:26.296657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266d64d-55fb-4229-a4e0-6a7216a00a34 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:26.359799 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:26.360665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:26.360951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:26.361228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] PATH_INFO: `/v3/projects/6a47e6cb906f45c38d04828a1c6ddddc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:26.362442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a47e6cb906f45c38d04828a1c6ddddc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:26.364369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:26.522431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a47e6cb906f45c38d04828a1c6ddddc', 'request_id': 'req-c341d8d4-8789-4bcb-a076-fa3551a58339'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:26.529334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a47e6cb906f45c38d04828a1c6ddddc', 'request_id': 'req-c341d8d4-8789-4bcb-a076-fa3551a58339'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:26.664815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a47e6cb906f45c38d04828a1c6ddddc', 'request_id': 'req-c341d8d4-8789-4bcb-a076-fa3551a58339'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:26.676102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] The token cache is being invalidate because project 6a47e6cb906f45c38d04828a1c6ddddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:26.676376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:26.677009 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c341d8d4-8789-4bcb-a076-fa3551a58339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:26.678631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1178/2338] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:26 2022] DELETE /identity/v3/projects/6a47e6cb906f45c38d04828a1c6ddddc => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:26.685413 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6221921-eda0-451c-9670-604dca51ada1 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:26.685929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6221921-eda0-451c-9670-604dca51ada1 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:26.686247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6221921-eda0-451c-9670-604dca51ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:26.708868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e6221921-eda0-451c-9670-604dca51ada1 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:26.710073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e6221921-eda0-451c-9670-604dca51ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:26.770382 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1161/2339] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:26.778259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0412748-0fbb-49ba-9340-2ee121f16fdf None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:26.778557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0412748-0fbb-49ba-9340-2ee121f16fdf None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:26.778839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0412748-0fbb-49ba-9340-2ee121f16fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:26.798326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-d0412748-0fbb-49ba-9340-2ee121f16fdf None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:26.799468 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d0412748-0fbb-49ba-9340-2ee121f16fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:26.808565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1179/2340] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:26.812903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6221921-eda0-451c-9670-604dca51ada1 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:26.823709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6221921-eda0-451c-9670-604dca51ada1 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:26.828351 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:26.829020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:26.829307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:26.829583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:26.830342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:26.830663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:26.832288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:26.845465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1162/2341] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:26.849989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0412748-0fbb-49ba-9340-2ee121f16fdf None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:26.859914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0412748-0fbb-49ba-9340-2ee121f16fdf None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:26.864816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:26.865475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:26.865760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:26.866044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:26.866780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:26.869353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:26.877722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:27.047155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1180/2342] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:26 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:27.048273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.058331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ccc55-8a24-4ccf-ad6e-b29d2dcc2594 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.121317 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.121995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.122328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.122610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.123307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:27.125309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:27.260678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1163/2343] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:27 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:27.264599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.276995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f08ba-0c3b-4a83-829a-0658d044867f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.281541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.282225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.282509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.282816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.283539 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:27.288003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:27.295668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1181/2344] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:02:27.299682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.310567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fedbce-b87f-43b8-a882-958c29cca9b5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.315135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c2ec0-36f6-4108-8066-6c76341feecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.315793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2ec0-36f6-4108-8066-6c76341feecc admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.316078 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2ec0-36f6-4108-8066-6c76341feecc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.316374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2ec0-36f6-4108-8066-6c76341feecc admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.317084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c2ec0-36f6-4108-8066-6c76341feecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:27.318765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c2ec0-36f6-4108-8066-6c76341feecc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:27.329399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1164/2345] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:27.333787 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.344011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaa997-7e75-493b-915b-696ed02da38a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.349114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.350029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.350335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.350612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] PATH_INFO: `/v3/projects/2d46cd473d6a41da9a1cdde9e16ffcb7/users/3cc49cdfd8894334af9b337e1267605e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.366951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d46cd473d6a41da9a1cdde9e16ffcb7, user_id=3cc49cdfd8894334af9b337e1267605e, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:27.370253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:27.465701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1182/2346] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:27 2022] PUT /identity/v3/projects/2d46cd473d6a41da9a1cdde9e16ffcb7/users/3cc49cdfd8894334af9b337e1267605e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:27.477352 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731f16-4b18-4893-8f56-9bc278f14dc2 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.477672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731f16-4b18-4893-8f56-9bc278f14dc2 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.478633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40731f16-4b18-4893-8f56-9bc278f14dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.506602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-40731f16-4b18-4893-8f56-9bc278f14dc2 None None] MFA Rules not processed for user `3cc49cdfd8894334af9b337e1267605e`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:27.507813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-40731f16-4b18-4893-8f56-9bc278f14dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:27.538073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:27.573135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1165/2347] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:27.577713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.639197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d50ea-f970-4fcb-9586-7943a619c4a7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.644002 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.644649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.644931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.645205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.645885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:27.647811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:27.763220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1183/2348] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:27 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:27.767132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731f16-4b18-4893-8f56-9bc278f14dc2 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.778117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40731f16-4b18-4893-8f56-9bc278f14dc2 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.836538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.837251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.837533 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.837805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.838545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:27.840841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:27.927875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1166/2349] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:27 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:27.932469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.942778 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e13e63-14b9-4a00-b879-a390e12ba14a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.947579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.948233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.948517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.948798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:27.949551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:27.951179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:27.961519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1184/2350] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:02:27.971097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:27.981737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f244563-19e7-41c3-8e82-a059e3efea7b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:27.986262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:27.987140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:27.987424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:27.987702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] PATH_INFO: `/v3/projects/95ffbc6d334540a589a242dbca219170/users/97d4a2bb789c40dba24495b83f5f2cdb/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:28.002617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ffbc6d334540a589a242dbca219170, user_id=97d4a2bb789c40dba24495b83f5f2cdb, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:28.005285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:28.100100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1167/2351] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:27 2022] PUT /identity/v3/projects/95ffbc6d334540a589a242dbca219170/users/97d4a2bb789c40dba24495b83f5f2cdb/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:28.105016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:28.115001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365d38-2c08-4e17-8757-9a04386eea61 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:28.175198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa93c4-5830-42fb-b1ce-c4ecf8ce8b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:28.175876 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa93c4-5830-42fb-b1ce-c4ecf8ce8b61 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:28.176164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa93c4-5830-42fb-b1ce-c4ecf8ce8b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:28.176443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa93c4-5830-42fb-b1ce-c4ecf8ce8b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:28.177180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa93c4-5830-42fb-b1ce-c4ecf8ce8b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:28.178833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa93c4-5830-42fb-b1ce-c4ecf8ce8b61 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:28.189463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1185/2352] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:28.193663 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:28.204488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad76ac-ef1a-4d9d-b52f-5071aa117e70 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:28.209420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:28.210349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:28.210639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:28.210915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] PATH_INFO: `/v3/projects/95ffbc6d334540a589a242dbca219170/users/97d4a2bb789c40dba24495b83f5f2cdb/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:28.213519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ffbc6d334540a589a242dbca219170, user_id=97d4a2bb789c40dba24495b83f5f2cdb, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:28.216250 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:28.227130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1168/2353] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:28 2022] PUT /identity/v3/projects/95ffbc6d334540a589a242dbca219170/users/97d4a2bb789c40dba24495b83f5f2cdb/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:28.237967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8cde-66bd-41d9-b05e-0fceca441527 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:28.238361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8cde-66bd-41d9-b05e-0fceca441527 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:28.238735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8cde-66bd-41d9-b05e-0fceca441527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:28.265183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-32fe8cde-66bd-41d9-b05e-0fceca441527 None None] MFA Rules not processed for user `97d4a2bb789c40dba24495b83f5f2cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:28.266462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-32fe8cde-66bd-41d9-b05e-0fceca441527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:28.279783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:28.339094 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1186/2354] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:28.350050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:28.355682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:28.409720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504836a0-3d0f-43c9-a86a-3adfb7cca4fd admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:28.413335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae9f58-c617-4875-afd8-5a02355d0108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:28.413975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae9f58-c617-4875-afd8-5a02355d0108 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:28.414308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae9f58-c617-4875-afd8-5a02355d0108 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:28.414573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae9f58-c617-4875-afd8-5a02355d0108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:28.424314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae9f58-c617-4875-afd8-5a02355d0108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:28.425769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae9f58-c617-4875-afd8-5a02355d0108 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:28.435205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1169/2355] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:28.969810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe8cde-66bd-41d9-b05e-0fceca441527 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:29.031722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe8cde-66bd-41d9-b05e-0fceca441527 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:29.036223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:29.037090 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:29.037370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:29.037644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] PATH_INFO: `/v3/users/3cc49cdfd8894334af9b337e1267605e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:29.038839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc49cdfd8894334af9b337e1267605e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:29.040957 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:29.284434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc49cdfd8894334af9b337e1267605e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.285036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc49cdfd8894334af9b337e1267605e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.292533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc49cdfd8894334af9b337e1267605e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.407901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc49cdfd8894334af9b337e1267605e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.421765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc49cdfd8894334af9b337e1267605e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:29.427732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1187/2356] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:28 2022] DELETE /identity/v3/users/3cc49cdfd8894334af9b337e1267605e => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:29.433394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:29.433768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:29.433941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:29.452770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:29.454059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:29.476346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:29.516078 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1170/2357] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:29.527177 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:29.529083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:29.567380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:29.592593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d1eb-ecc6-4112-abf3-d42c415af54c admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:29.596421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:29.597110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:29.597396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:29.597674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:29.606731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:29.608163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:29.617392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1188/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:30.278440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.279864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:30.292204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ddc71-ce7e-42f9-a7e9-0ba8fb1d8195 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.296959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.297832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.298116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.298443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] PATH_INFO: `/v3/projects/2d46cd473d6a41da9a1cdde9e16ffcb7` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.299613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d46cd473d6a41da9a1cdde9e16ffcb7)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.301559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.470678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d46cd473d6a41da9a1cdde9e16ffcb7', 'request_id': 'req-712b7390-b81a-43ed-94e9-3ac931fb56fc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.550114 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d46cd473d6a41da9a1cdde9e16ffcb7', 'request_id': 'req-712b7390-b81a-43ed-94e9-3ac931fb56fc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.558268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d46cd473d6a41da9a1cdde9e16ffcb7', 'request_id': 'req-712b7390-b81a-43ed-94e9-3ac931fb56fc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.572477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] The token cache is being invalidate because project 2d46cd473d6a41da9a1cdde9e16ffcb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:30.572768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.575209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.575209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1171/2359] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:30 2022] DELETE /identity/v3/projects/2d46cd473d6a41da9a1cdde9e16ffcb7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:30.579232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:30.589457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9bea4-cc63-4012-b2ba-cbb13e82f47a service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:30.651380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:30.652618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:30.652901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:30.653174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] PATH_INFO: `/v3/users/97d4a2bb789c40dba24495b83f5f2cdb` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:30.654392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d4a2bb789c40dba24495b83f5f2cdb)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:30.656320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:30.937275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d4a2bb789c40dba24495b83f5f2cdb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.938306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d4a2bb789c40dba24495b83f5f2cdb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:30.945189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d4a2bb789c40dba24495b83f5f2cdb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.064560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d4a2bb789c40dba24495b83f5f2cdb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.078874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-aaa14a89-c9cf-457f-b902-e514e49b6470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d4a2bb789c40dba24495b83f5f2cdb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.085423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1189/2360] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:30 2022] DELETE /identity/v3/users/97d4a2bb789c40dba24495b83f5f2cdb => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:31.605306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:31.608421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:31.623859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7390-b81a-43ed-94e9-3ac931fb56fc admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:31.683164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:31.684036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:31.684326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:31.684607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] PATH_INFO: `/v3/projects/95ffbc6d334540a589a242dbca219170` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:31.685796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ffbc6d334540a589a242dbca219170)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:31.687772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:31.814611 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ffbc6d334540a589a242dbca219170', 'request_id': 'req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.817166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:31.881759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ffbc6d334540a589a242dbca219170', 'request_id': 'req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.888522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ffbc6d334540a589a242dbca219170', 'request_id': 'req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.900387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] The token cache is being invalidate because project 95ffbc6d334540a589a242dbca219170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:31.900676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.901323 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-22b487a0-a651-4d84-9a37-ae9f29d4fdb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:31.902910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1172/2361] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:31 2022] DELETE /identity/v3/projects/95ffbc6d334540a589a242dbca219170 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:31.909222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc66d-6ed3-4df6-a4ed-380b33ee94eb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:31.909734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc66d-6ed3-4df6-a4ed-380b33ee94eb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:31.909845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc66d-6ed3-4df6-a4ed-380b33ee94eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:31.940966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-311cc66d-6ed3-4df6-a4ed-380b33ee94eb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:31.940966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-311cc66d-6ed3-4df6-a4ed-380b33ee94eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:31.980722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:31.995971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1190/2362] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:32.004789 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9e5e5-038d-4091-a98f-b997ad167e49 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.015514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9e5e5-038d-4091-a98f-b997ad167e49 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.015514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9e5e5-038d-4091-a98f-b997ad167e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.024027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e6e9e5e5-038d-4091-a98f-b997ad167e49 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:32.025418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e6e9e5e5-038d-4091-a98f-b997ad167e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:32.033561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1173/2363] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:32.037675 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cc66d-6ed3-4df6-a4ed-380b33ee94eb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.047361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cc66d-6ed3-4df6-a4ed-380b33ee94eb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.052039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:32.052697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.052982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.053263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.053990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:32.054346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:32.056492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:32.068124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1191/2364] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:32.073407 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9e5e5-038d-4091-a98f-b997ad167e49 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.082478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9e5e5-038d-4091-a98f-b997ad167e49 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.091269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:32.091269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.091269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.091269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.091648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:32.091648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:32.235779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1174/2365] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:32 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:32.236955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.248070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b8c06-a1f8-4670-a5af-9815eeff3719 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.315161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:32.315842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.316174 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.316489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.317251 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:32.319221 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:32.419427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1192/2366] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:32 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:32.423322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.434481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ef802-a96e-4e4b-9179-5110a3888006 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.439069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:32.439726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.440010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.440271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.440990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:32.442692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:32.452973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1175/2367] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:32.456880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.467017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f71fcb-bf84-48e4-b8f3-044ef45a7c3b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.471939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7b3d7-f20a-48c3-bd81-4e23e0caf446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:32.472598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b3d7-f20a-48c3-bd81-4e23e0caf446 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.472879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b3d7-f20a-48c3-bd81-4e23e0caf446 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.473160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b3d7-f20a-48c3-bd81-4e23e0caf446 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.473888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7b3d7-f20a-48c3-bd81-4e23e0caf446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:32.475526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7b3d7-f20a-48c3-bd81-4e23e0caf446 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:32.486402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1193/2368] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:02:32.490542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.501796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6c065-310c-43b6-8bc3-df69dd26bd5c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.506554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:32.516236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.516236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.516236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] PATH_INFO: `/v3/projects/4ed2986a0d14437198925a37601a5755/users/82a40f2e63e24d37ae14796a9cd08e74/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.522768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed2986a0d14437198925a37601a5755, user_id=82a40f2e63e24d37ae14796a9cd08e74, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:32.525491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:32.619497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1176/2369] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:32 2022] PUT /identity/v3/projects/4ed2986a0d14437198925a37601a5755/users/82a40f2e63e24d37ae14796a9cd08e74/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:32.629853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80609e-5a7a-4efc-9140-295d4cb4ff9f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.630235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80609e-5a7a-4efc-9140-295d4cb4ff9f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.630605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80609e-5a7a-4efc-9140-295d4cb4ff9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.658934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-9b80609e-5a7a-4efc-9140-295d4cb4ff9f None None] MFA Rules not processed for user `82a40f2e63e24d37ae14796a9cd08e74`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:32.660183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9b80609e-5a7a-4efc-9140-295d4cb4ff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:32.724643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1194/2370] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:32.730328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.803597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1ec6-1bd8-4bd2-9cf9-5ed539e3bcdb admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.808781 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:32.809838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:32.810129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:32.810429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:32.811111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:32.813045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:32.928796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1177/2371] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:32 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:32.933470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80609e-5a7a-4efc-9140-295d4cb4ff9f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:32.944223 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80609e-5a7a-4efc-9140-295d4cb4ff9f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:32.994571 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:33.008437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:33.009797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:33.010264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:33.010545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:33.011267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:33.013231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:33.105059 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1195/2372] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:32 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:33.109140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:33.119532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef3cbb-79fe-43a1-8224-89da77e2c07e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:33.124947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:33.125609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:33.125890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:33.126166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:33.126922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:33.128553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:33.140051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1178/2373] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:33.148129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:33.159049 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74b02f-6641-4fa6-aded-c0c86a43920a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:33.163575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:33.164450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:33.164758 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:33.165615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] PATH_INFO: `/v3/projects/ce330a7919064588a0884ea84dc52ecb/users/fb2378eb774f48cf8e2021a18799d29a/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:33.180314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce330a7919064588a0884ea84dc52ecb, user_id=fb2378eb774f48cf8e2021a18799d29a, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:33.183292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:33.253185 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1196/2374] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:33 2022] PUT /identity/v3/projects/ce330a7919064588a0884ea84dc52ecb/users/fb2378eb774f48cf8e2021a18799d29a/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:33.257299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:33.268347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2424fc-f719-4089-8c6f-49d1bdd8a395 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:33.320494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:33.331612 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6079854-d299-4c5e-8b53-668d681c5ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:33.332795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079854-d299-4c5e-8b53-668d681c5ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:33.332892 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079854-d299-4c5e-8b53-668d681c5ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:33.335252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079854-d299-4c5e-8b53-668d681c5ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:33.335252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6079854-d299-4c5e-8b53-668d681c5ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:33.335629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6079854-d299-4c5e-8b53-668d681c5ad5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:33.349246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1179/2375] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:33.350966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:33.363169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e997c6-2aa4-4f5d-8c71-82e7e15446e5 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:33.369434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:33.371569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:33.371569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:33.371569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] PATH_INFO: `/v3/projects/ce330a7919064588a0884ea84dc52ecb/users/fb2378eb774f48cf8e2021a18799d29a/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:33.373213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce330a7919064588a0884ea84dc52ecb, user_id=fb2378eb774f48cf8e2021a18799d29a, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:33.376268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:33.390538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1197/2376] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:33 2022] PUT /identity/v3/projects/ce330a7919064588a0884ea84dc52ecb/users/fb2378eb774f48cf8e2021a18799d29a/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:33.400993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf820a-a6b2-48a8-9d6d-2303efcbce62 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:33.401283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf820a-a6b2-48a8-9d6d-2303efcbce62 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:33.401567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf820a-a6b2-48a8-9d6d-2303efcbce62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:33.428439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9abf820a-a6b2-48a8-9d6d-2303efcbce62 None None] MFA Rules not processed for user `fb2378eb774f48cf8e2021a18799d29a`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:33.429726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9abf820a-a6b2-48a8-9d6d-2303efcbce62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:33.514954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1180/2377] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:33.522480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:33.524238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:33.600081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdad23-5931-478b-b149-aec33d8df534 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:33.605600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1ab58-3235-4d2a-9640-1497f4310d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:33.606484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ab58-3235-4d2a-9640-1497f4310d0f service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:33.606944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ab58-3235-4d2a-9640-1497f4310d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:33.607413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1ab58-3235-4d2a-9640-1497f4310d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:33.676104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ab58-3235-4d2a-9640-1497f4310d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:33.677628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1ab58-3235-4d2a-9640-1497f4310d0f service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:33.687265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1198/2378] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:37.138271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf820a-a6b2-48a8-9d6d-2303efcbce62 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:37.201614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf820a-a6b2-48a8-9d6d-2303efcbce62 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:37.206155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:37.207048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:37.207331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:37.207614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] PATH_INFO: `/v3/users/82a40f2e63e24d37ae14796a9cd08e74` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:37.208810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a40f2e63e24d37ae14796a9cd08e74)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:37.210810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:37.460572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a40f2e63e24d37ae14796a9cd08e74'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:37.461224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a40f2e63e24d37ae14796a9cd08e74'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:37.462076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a40f2e63e24d37ae14796a9cd08e74'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:37.475397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a40f2e63e24d37ae14796a9cd08e74'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:37.563826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a40f2e63e24d37ae14796a9cd08e74'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:37.566800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:37.578654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1181/2379] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:37 2022] DELETE /identity/v3/users/82a40f2e63e24d37ae14796a9cd08e74 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:37.584108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:37.584623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:37.584731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:37.607570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:37.608870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:37.659896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:37.686347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1199/2380] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:37.686913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:37.688441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:37.751193 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056eac-ed12-4b8d-9df4-9a4349ef365c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:37.754899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:37.755531 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:37.755815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:37.756090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:37.765719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:37.767196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:37.776774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1182/2381] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:38.277811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:38.284011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:38.291537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397545a-bbdd-4c1e-bc35-ef4b3fca48f1 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:38.296336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:38.297304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:38.297654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:38.298000 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] PATH_INFO: `/v3/projects/4ed2986a0d14437198925a37601a5755` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:38.299264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed2986a0d14437198925a37601a5755)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:38.301266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:38.499412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed2986a0d14437198925a37601a5755', 'request_id': 'req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:38.505894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed2986a0d14437198925a37601a5755', 'request_id': 'req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:38.613426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed2986a0d14437198925a37601a5755', 'request_id': 'req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:38.625649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] The token cache is being invalidate because project 4ed2986a0d14437198925a37601a5755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:38.625940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:38.626623 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-719a8193-355f-4e73-b9b8-5ff8cd53d9a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:38.628180 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1200/2382] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:38 2022] DELETE /identity/v3/projects/4ed2986a0d14437198925a37601a5755 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:38.632208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:38.642916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431aca5-3c4e-45dd-8117-e7b9c778f3e3 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:38.718557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:38.719448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:38.719729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:38.720091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] PATH_INFO: `/v3/users/fb2378eb774f48cf8e2021a18799d29a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:38.721569 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2378eb774f48cf8e2021a18799d29a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:38.723667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:39.086914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2378eb774f48cf8e2021a18799d29a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:39.087697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2378eb774f48cf8e2021a18799d29a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:39.088388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2378eb774f48cf8e2021a18799d29a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:39.102444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2378eb774f48cf8e2021a18799d29a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:39.206819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2378eb774f48cf8e2021a18799d29a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:39.220998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1183/2383] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:02:38 2022] DELETE /identity/v3/users/fb2378eb774f48cf8e2021a18799d29a => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:39.592672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:39.592960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:39.593240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:39.613331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:39.614691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:39.709097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1201/2384] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:39.746847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:39.748344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:39.766969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:39.818128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cd79b-959a-4846-bcf4-e5b5e0f84373 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:39.823825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:39.824709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:39.824992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:39.825270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] PATH_INFO: `/v3/projects/ce330a7919064588a0884ea84dc52ecb` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:39.826588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce330a7919064588a0884ea84dc52ecb)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:39.828901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:40.011878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:40.014214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:40.103561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2e41d-9a42-4e1a-9081-ed1903f13cd7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:40.110060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94ae9b-a700-4de5-9188-4da4928681b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:40.110848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ae9b-a700-4de5-9188-4da4928681b5 service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:40.111376 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ae9b-a700-4de5-9188-4da4928681b5 service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:40.111746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ae9b-a700-4de5-9188-4da4928681b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:40.122413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94ae9b-a700-4de5-9188-4da4928681b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:40.124248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94ae9b-a700-4de5-9188-4da4928681b5 service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:40.136562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1202/2385] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:02:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 15:02:40.154817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce330a7919064588a0884ea84dc52ecb', 'request_id': 'req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:40.344791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce330a7919064588a0884ea84dc52ecb', 'request_id': 'req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:40.353524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce330a7919064588a0884ea84dc52ecb', 'request_id': 'req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:40.366315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] The token cache is being invalidate because project ce330a7919064588a0884ea84dc52ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:02:40.366992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:40.368652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-42c50ca0-d5a0-4b41-b6f0-dddb18eb036f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:02:40.370669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1184/2386] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:02:39 2022] DELETE /identity/v3/projects/ce330a7919064588a0884ea84dc52ecb => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:02:40.376633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a65b5-db72-4f25-a736-0c101e2e1707 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:40.376951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a65b5-db72-4f25-a736-0c101e2e1707 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:40.377262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a65b5-db72-4f25-a736-0c101e2e1707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:40.406657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-525a65b5-db72-4f25-a736-0c101e2e1707 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:40.407966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-525a65b5-db72-4f25-a736-0c101e2e1707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:40.424456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:40.491959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1203/2387] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:02:40.500079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7424-67cc-4490-814a-2d4b424902c3 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:40.500562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7424-67cc-4490-814a-2d4b424902c3 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:40.501036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7424-67cc-4490-814a-2d4b424902c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:40.522862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b0dc7424-67cc-4490-814a-2d4b424902c3 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:40.522862 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b0dc7424-67cc-4490-814a-2d4b424902c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:40.547086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1185/2388] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:40.549645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a65b5-db72-4f25-a736-0c101e2e1707 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:40.562060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a65b5-db72-4f25-a736-0c101e2e1707 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:40.567587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:40.568606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:40.568889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:40.569166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:40.569911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:02:40.570249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:40.572055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:40.592913 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1204/2389] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:02:40.593850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc7424-67cc-4490-814a-2d4b424902c3 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:40.606476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc7424-67cc-4490-814a-2d4b424902c3 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:40.612853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:40.613584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:40.613868 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:40.614145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:40.614884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:40.616947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:40.836846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1186/2390] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:02:40 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:02:40.841212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:40.852797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b0c3b-f393-4133-bb29-8cdad3302047 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:40.986950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:40.987801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:40.988155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:40.988499 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:40.989276 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:40.994551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:41.163314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:41.164235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1205/2391] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:02:40 2022] POST /identity/v3/users => generated 285 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:02:41.177436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd9731-d750-4adb-930d-7623912dfe20 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:41.183031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:41.184301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:41.184802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:41.185465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:41.186484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:41.188534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:41.224293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:41.235894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1187/2392] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:02:41.239849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a3002-6366-4a9b-9a6b-c7c9df61764b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:41.249915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2f7ac-861f-4cc5-9872-ddaa6f55091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:41.250859 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2f7ac-861f-4cc5-9872-ddaa6f55091a admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:41.251240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2f7ac-861f-4cc5-9872-ddaa6f55091a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:41.251621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2f7ac-861f-4cc5-9872-ddaa6f55091a admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:41.252527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2f7ac-861f-4cc5-9872-ddaa6f55091a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:41.254322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2f7ac-861f-4cc5-9872-ddaa6f55091a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:41.286164 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1206/2393] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:02:41.301400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:41.305286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d5296-1b99-4f96-8a00-a0ad4bddd383 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:41.311964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:41.313241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:41.313622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:41.314091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] PATH_INFO: `/v3/projects/c398de11386144d9abe8d595d561ed91/users/5c3e95108c564776969a86f3fe426915/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:41.355928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c398de11386144d9abe8d595d561ed91, user_id=5c3e95108c564776969a86f3fe426915, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:41.360323 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:41.588597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1188/2394] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:02:41 2022] PUT /identity/v3/projects/c398de11386144d9abe8d595d561ed91/users/5c3e95108c564776969a86f3fe426915/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:02:41.590453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6042-f2a4-4ddc-ac01-d54d2f7547d4 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:41.590980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6042-f2a4-4ddc-ac01-d54d2f7547d4 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:41.591506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6042-f2a4-4ddc-ac01-d54d2f7547d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:41.628331 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-61cf6042-f2a4-4ddc-ac01-d54d2f7547d4 None None] MFA Rules not processed for user `5c3e95108c564776969a86f3fe426915`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:02:41.629871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-61cf6042-f2a4-4ddc-ac01-d54d2f7547d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:41.775880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1207/2395] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:02:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:02:41.789852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:02:41.791617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:02:41.882660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:41.912552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2737be-3b21-4cfd-9f70-806175e10748 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:02:41.917331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e145d-f956-4a69-b181-3ea4fa6a2f86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:02:41.918106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e145d-f956-4a69-b181-3ea4fa6a2f86 service glance] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:02:41.918476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e145d-f956-4a69-b181-3ea4fa6a2f86 service glance] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:02:41.918824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e145d-f956-4a69-b181-3ea4fa6a2f86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:02:41.934978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e145d-f956-4a69-b181-3ea4fa6a2f86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:02:41.938759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e145d-f956-4a69-b181-3ea4fa6a2f86 service glance] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:02:41.953809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1189/2396] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:02:51.770387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:02:52.069720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=8 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:02.589064 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2601b-352e-4169-8763-f4d47bca1b84 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:02.589736 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2601b-352e-4169-8763-f4d47bca1b84 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:02.589736 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2601b-352e-4169-8763-f4d47bca1b84 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:02.591086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1208/2397] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 15:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:03:02.597443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054fdbb-e692-4fb7-9d78-d019aca3e94e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:02.597863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054fdbb-e692-4fb7-9d78-d019aca3e94e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:02.598014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054fdbb-e692-4fb7-9d78-d019aca3e94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:02.617634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7054fdbb-e692-4fb7-9d78-d019aca3e94e None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:02.618966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7054fdbb-e692-4fb7-9d78-d019aca3e94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:02.701255 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1190/2398] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:02.713233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2601b-352e-4169-8763-f4d47bca1b84 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:02.715134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-34f2601b-352e-4169-8763-f4d47bca1b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:02.729879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2601b-352e-4169-8763-f4d47bca1b84 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:02.733408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d91f4-47b7-48d2-94dd-3dcd10f58062 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:02.734041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d91f4-47b7-48d2-94dd-3dcd10f58062 service glance] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:02.734354 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d91f4-47b7-48d2-94dd-3dcd10f58062 service glance] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:02.734627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d91f4-47b7-48d2-94dd-3dcd10f58062 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:02.745722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d91f4-47b7-48d2-94dd-3dcd10f58062 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:02.747112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d91f4-47b7-48d2-94dd-3dcd10f58062 service glance] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:02.761554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1209/2399] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:03.064476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806fe1d-b112-4dcc-94b6-9c1aeaa83f8a None None] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:03.064772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806fe1d-b112-4dcc-94b6-9c1aeaa83f8a None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:03.065058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806fe1d-b112-4dcc-94b6-9c1aeaa83f8a None None] PATH_INFO: `/` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:03.066516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1191/2400] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 15:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:03:03.071330 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8398-0c37-416b-8d2b-a4d0f07299f8 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:03.071622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8398-0c37-416b-8d2b-a4d0f07299f8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:03.071975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8398-0c37-416b-8d2b-a4d0f07299f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:03.091334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-899a8398-0c37-416b-8d2b-a4d0f07299f8 None None] MFA Rules not processed for user `bead604b09be482c906a97b53bb03d69`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:03.092555 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-899a8398-0c37-416b-8d2b-a4d0f07299f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:03.151713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1210/2401] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 15:03:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:03.164275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806fe1d-b112-4dcc-94b6-9c1aeaa83f8a None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:03.166005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3806fe1d-b112-4dcc-94b6-9c1aeaa83f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:03.179809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806fe1d-b112-4dcc-94b6-9c1aeaa83f8a None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:03.183332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ce8a2-c924-4591-837b-7fcaf31d9be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:03.184002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ce8a2-c924-4591-837b-7fcaf31d9be2 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:03.184285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ce8a2-c924-4591-837b-7fcaf31d9be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:03.184563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ce8a2-c924-4591-837b-7fcaf31d9be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:03.191630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ce8a2-c924-4591-837b-7fcaf31d9be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:03.193007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ce8a2-c924-4591-837b-7fcaf31d9be2 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:03.201828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1192/2402] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:03.296389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a3f80-ec6d-4c21-b247-1e7a842e1f74 None None] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:03.296701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a3f80-ec6d-4c21-b247-1e7a842e1f74 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:03.296980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a3f80-ec6d-4c21-b247-1e7a842e1f74 None None] PATH_INFO: `/` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:03.298380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1211/2403] 10.222.0.23 () {54 vars in 921 bytes} [Wed Jun 15 15:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 15:03:03.303378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec44aa-3b80-4118-8717-fc2536b7332c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:03.304186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec44aa-3b80-4118-8717-fc2536b7332c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:03.304461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec44aa-3b80-4118-8717-fc2536b7332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:03.323415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-93ec44aa-3b80-4118-8717-fc2536b7332c None None] MFA Rules not processed for user `ac6a657099c14f598d5becd5ab72ee84`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:03.324632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-93ec44aa-3b80-4118-8717-fc2536b7332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:03.333878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1193/2404] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Jun 15 15:03:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:03.344316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a3f80-ec6d-4c21-b247-1e7a842e1f74 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:03.355504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a3f80-ec6d-4c21-b247-1e7a842e1f74 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:03.359239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:03.359864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:03.360161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:03.360433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:03.371906 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:03.373332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:03.384401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1212/2405] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:09.010083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c72114-ab9a-4465-b346-1022653f2eae None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:09.010603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c72114-ab9a-4465-b346-1022653f2eae None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:09.010800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c72114-ab9a-4465-b346-1022653f2eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:09.029592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f6c72114-ab9a-4465-b346-1022653f2eae None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:09.030745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f6c72114-ab9a-4465-b346-1022653f2eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:09.087693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1194/2406] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 15:03:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:09.099855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:09.101349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:09.175036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352eb430-46a5-4a4d-a13e-5171507968dc service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:09.179990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159901d-3009-40b5-8405-9a38d74b5538 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:09.181967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159901d-3009-40b5-8405-9a38d74b5538 service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:09.182274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159901d-3009-40b5-8405-9a38d74b5538 service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:09.182550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159901d-3009-40b5-8405-9a38d74b5538 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:09.193662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159901d-3009-40b5-8405-9a38d74b5538 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:09.196751 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159901d-3009-40b5-8405-9a38d74b5538 service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:09.207984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1213/2407] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:03:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:03:19.080240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=65 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:19.204599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=67 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:24.398024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c72114-ab9a-4465-b346-1022653f2eae None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:24.459570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c72114-ab9a-4465-b346-1022653f2eae None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:24.464264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:24.465271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:24.465581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:24.465852 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] PATH_INFO: `/v3/users/5c3e95108c564776969a86f3fe426915` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:24.467076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3e95108c564776969a86f3fe426915)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:24.469002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:24.658570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3e95108c564776969a86f3fe426915'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:24.659219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3e95108c564776969a86f3fe426915'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:24.659835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3e95108c564776969a86f3fe426915'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:24.672041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3e95108c564776969a86f3fe426915'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:24.719252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3e95108c564776969a86f3fe426915'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:24.732819 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1195/2408] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:24 2022] DELETE /identity/v3/users/5c3e95108c564776969a86f3fe426915 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:24.737419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:24.737882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:24.738006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:24.756239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:24.757423 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:24.820241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1214/2409] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:24.830019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:24.831481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:24.889032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd05b1-95be-4462-a190-01f1d10eb622 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:24.892666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5c63c-22fa-486f-aacb-b18fe930f5f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:24.893306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5c63c-22fa-486f-aacb-b18fe930f5f1 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:24.893588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5c63c-22fa-486f-aacb-b18fe930f5f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:24.893861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5c63c-22fa-486f-aacb-b18fe930f5f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:24.903215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5c63c-22fa-486f-aacb-b18fe930f5f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:24.904665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5c63c-22fa-486f-aacb-b18fe930f5f1 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:24.913979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1196/2410] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:25.310400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:25.312207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:25.333025 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f735b5-ce77-42e6-8cd3-d4d66569f4bb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:25.338644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:25.339613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.340595 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.340958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.341326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.341546 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] PATH_INFO: `/v3/projects/c398de11386144d9abe8d595d561ed91` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.341649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.342901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c398de11386144d9abe8d595d561ed91)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:25.345397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:25.370561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:25.376447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:25.397782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:25.440046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1197/2411] 10.222.0.23 () {58 vars in 967 bytes} [Wed Jun 15 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:25.452779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:25.454204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:25.496508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c398de11386144d9abe8d595d561ed91', 'request_id': 'req-f9783692-7aab-47c6-bbd2-efc367e8ecf6'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:25.502924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c398de11386144d9abe8d595d561ed91', 'request_id': 'req-f9783692-7aab-47c6-bbd2-efc367e8ecf6'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:25.525272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cfa94-afe7-441d-8390-606de1e541e9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:25.583919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc245a-c82d-4888-8499-e382b794641e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:25.584484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc245a-c82d-4888-8499-e382b794641e service swift] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.584771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc245a-c82d-4888-8499-e382b794641e service swift] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.585050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc245a-c82d-4888-8499-e382b794641e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.641006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc245a-c82d-4888-8499-e382b794641e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:25.642657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc245a-c82d-4888-8499-e382b794641e service swift] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:25.650736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1198/2412] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:03:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 15:03:25.657373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c398de11386144d9abe8d595d561ed91', 'request_id': 'req-f9783692-7aab-47c6-bbd2-efc367e8ecf6'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:25.682199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] The token cache is being invalidate because project c398de11386144d9abe8d595d561ed91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:25.682199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:25.682199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f9783692-7aab-47c6-bbd2-efc367e8ecf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:25.682199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1215/2413] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:25 2022] DELETE /identity/v3/projects/c398de11386144d9abe8d595d561ed91 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:25.682813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0db44-3ff5-4d8e-8c35-a99aea0321cd None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.682813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0db44-3ff5-4d8e-8c35-a99aea0321cd None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.682813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0db44-3ff5-4d8e-8c35-a99aea0321cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.698133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-32e0db44-3ff5-4d8e-8c35-a99aea0321cd None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:25.700159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-32e0db44-3ff5-4d8e-8c35-a99aea0321cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:25.758505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1199/2414] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:25.821233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b42da-037f-4d75-a21d-7fbbe0e3cd4f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.821779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b42da-037f-4d75-a21d-7fbbe0e3cd4f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.821907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b42da-037f-4d75-a21d-7fbbe0e3cd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.827685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:25.841492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6d4b42da-037f-4d75-a21d-7fbbe0e3cd4f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:25.842664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6d4b42da-037f-4d75-a21d-7fbbe0e3cd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:25.850947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1216/2415] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:25.855256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0db44-3ff5-4d8e-8c35-a99aea0321cd None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:25.869957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0db44-3ff5-4d8e-8c35-a99aea0321cd None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:25.872188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:25.872188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.872188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.872909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.873272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:03:25.873584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:25.875146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:25.888212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1200/2416] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:03:25.892510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b42da-037f-4d75-a21d-7fbbe0e3cd4f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:25.901753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b42da-037f-4d75-a21d-7fbbe0e3cd4f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:25.906909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:25.907855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:25.908138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:25.908416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:25.909108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:25.911100 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:26.019483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1217/2417] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:25 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:03:26.023446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:26.033648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47867e30-e5be-45e3-8506-c9aa08a4e6ae admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:26.090761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:26.091877 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:26.092144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:26.092421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:26.093111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:26.095600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:26.189330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1201/2418] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:26 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:26.193464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:26.204015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896af0d0-31ef-4bd5-8755-b772bbad104e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:26.208723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:26.209383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:26.209654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:26.209931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:26.210695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:26.212396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:26.223942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1218/2419] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:03:26.229227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:26.239734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446b8e7-a32a-44e7-b36d-e6e077afc983 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:26.244504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73431e2-00d9-4e5a-9ba9-3109f2301e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:26.245200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73431e2-00d9-4e5a-9ba9-3109f2301e5c admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:26.245479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73431e2-00d9-4e5a-9ba9-3109f2301e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:26.245786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73431e2-00d9-4e5a-9ba9-3109f2301e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:26.247123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73431e2-00d9-4e5a-9ba9-3109f2301e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:26.248750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73431e2-00d9-4e5a-9ba9-3109f2301e5c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:26.259414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1202/2420] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:03:26.263988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:26.273817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee6a8f-502c-4dea-b9b7-ca73f2759b72 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:26.279004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:26.279902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:26.280186 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:26.280460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] PATH_INFO: `/v3/projects/019a9c2e2943403ab0700a96af732826/users/833efc71510e4bf8a5418671026a1b29/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:26.296512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] RBAC: Authorizing `identity:create_grant(project_id=019a9c2e2943403ab0700a96af732826, user_id=833efc71510e4bf8a5418671026a1b29, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:26.299332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:26.370836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1219/2421] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:26 2022] PUT /identity/v3/projects/019a9c2e2943403ab0700a96af732826/users/833efc71510e4bf8a5418671026a1b29/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:26.376528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee9da-ea6f-48ff-a1cf-f3694cfe5928 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:26.376815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee9da-ea6f-48ff-a1cf-f3694cfe5928 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:26.377097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee9da-ea6f-48ff-a1cf-f3694cfe5928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:26.393972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-72aee9da-ea6f-48ff-a1cf-f3694cfe5928 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:26.395196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-72aee9da-ea6f-48ff-a1cf-f3694cfe5928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:26.460061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1203/2422] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:26.472338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:26.474106 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:26.497170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:26.534673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3f67-eeef-4e80-9fc7-23c5ac22236c admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:26.538415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:26.539070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:26.539351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:26.539627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:26.550418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:26.551891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:26.563804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1220/2423] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:03:32.431105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd305-73ab-414f-95e2-7650929c888d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:32.431741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd305-73ab-414f-95e2-7650929c888d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:32.431741 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd305-73ab-414f-95e2-7650929c888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:32.460895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a1acd305-73ab-414f-95e2-7650929c888d None None] MFA Rules not processed for user `833efc71510e4bf8a5418671026a1b29`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:32.462116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a1acd305-73ab-414f-95e2-7650929c888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:32.468780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:32.558306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1204/2424] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:32.563712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:32.579562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e7e5c-4396-4fae-b6df-9178eca4448d service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:32.584710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:32.585393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:32.585704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:32.585986 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:32.588642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:32.591259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:32.730700 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1221/2425] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:32 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:32.734744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acd305-73ab-414f-95e2-7650929c888d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:32.746685 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acd305-73ab-414f-95e2-7650929c888d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:32.815284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:32.815975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:32.816373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:32.816745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:32.817545 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:32.819716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:32.959897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1205/2426] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:32 2022] POST /identity/v3/users => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:32.963888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:32.975373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12751263-db2b-4fe4-9f3b-288f82f55930 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:32.980086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:32.981372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:32.981372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:32.981372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:32.982060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:32.983728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:32.994846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1222/2427] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:32.999120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:33.009414 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694686-2a6b-4711-a5d4-d97bcea280cd admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:33.015226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:33.016137 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:33.016386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:33.016657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] PATH_INFO: `/v3/projects/66132029bce54dccb5d9a8a4ebe01d99/users/ee0d661b6afe4de3b813e19e1597fcb3/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:33.032579 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66132029bce54dccb5d9a8a4ebe01d99, user_id=ee0d661b6afe4de3b813e19e1597fcb3, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:33.035357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:33.138624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1206/2428] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:32 2022] PUT /identity/v3/projects/66132029bce54dccb5d9a8a4ebe01d99/users/ee0d661b6afe4de3b813e19e1597fcb3/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:03:33.148449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:33.159245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd5701-41a6-4619-b271-9756cea99c0d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:33.221649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d28f75-0ceb-439f-84de-69b6eb701a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:33.222451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d28f75-0ceb-439f-84de-69b6eb701a32 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:33.222821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d28f75-0ceb-439f-84de-69b6eb701a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:33.223182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d28f75-0ceb-439f-84de-69b6eb701a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:33.223967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d28f75-0ceb-439f-84de-69b6eb701a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:33.225674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d28f75-0ceb-439f-84de-69b6eb701a32 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:33.236706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1223/2429] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:33.240880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:33.252115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d95a4-a9b1-435e-a030-670ad49cd308 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:33.257347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:33.258344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:33.258690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:33.259033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] PATH_INFO: `/v3/projects/66132029bce54dccb5d9a8a4ebe01d99/users/ee0d661b6afe4de3b813e19e1597fcb3/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:33.261080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66132029bce54dccb5d9a8a4ebe01d99, user_id=ee0d661b6afe4de3b813e19e1597fcb3, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:33.264756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:33.276188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1207/2430] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:33 2022] PUT /identity/v3/projects/66132029bce54dccb5d9a8a4ebe01d99/users/ee0d661b6afe4de3b813e19e1597fcb3/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:03:40.195332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261e070-db6f-483e-87ab-199742de1d6e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:40.196262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261e070-db6f-483e-87ab-199742de1d6e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:40.196784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261e070-db6f-483e-87ab-199742de1d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:40.232550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5261e070-db6f-483e-87ab-199742de1d6e None None] MFA Rules not processed for user `ee0d661b6afe4de3b813e19e1597fcb3`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:40.233860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5261e070-db6f-483e-87ab-199742de1d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:40.334911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1224/2431] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:40.354195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:40.366977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:40.434320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb1548-8fa1-465f-81b8-34871bc95173 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:40.439976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:40.440649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:40.441066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:40.441518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:40.456543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:40.510653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:40.512939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:40.522689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1208/2432] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:40.851187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9979-b761-4e29-bc0e-db4ddb47274f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:40.851449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9979-b761-4e29-bc0e-db4ddb47274f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:40.851729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef9979-b761-4e29-bc0e-db4ddb47274f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:40.872885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-cdef9979-b761-4e29-bc0e-db4ddb47274f None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:40.874268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cdef9979-b761-4e29-bc0e-db4ddb47274f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:40.883250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:40.943727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1225/2433] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 15:03:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:40.975648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:40.977333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:41.112314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540071f6-2072-48e7-82ac-af28513f578a service glance] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:41.117796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:41.118563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:41.118966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:41.119320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:41.133301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:41.135153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:41.145698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1209/2434] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:03:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:03:45.959367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef9979-b761-4e29-bc0e-db4ddb47274f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:46.094766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef9979-b761-4e29-bc0e-db4ddb47274f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:46.099858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:46.100730 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:46.101077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:46.101442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:46.103636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:46.217530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:46.219314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:46.234717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1226/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:47.643516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:47.647870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:47.671121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb98b30-af76-44ec-9dac-f14332148436 service swift] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:47.678016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:47.679461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:47.681485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:47.681945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:47.692283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:47.715033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:47.716795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:47.736713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1210/2436] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:51.902387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:51.903989 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:52.052825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33e6d-5ec7-49b3-a82b-b07d92e633bc service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:52.059554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:52.060242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:52.060916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:52.061202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:52.064863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:52.083544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:52.086604 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:52.089256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:52.112078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1227/2437] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:03:52.114772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:52.116491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:52.139310 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9e50a-1d20-426a-bf87-5d36debc0c14 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:52.152775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:52.153688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:52.156476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:52.156772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] PATH_INFO: `/v3/users/e367e391e68a477d861a515f846a8dfa` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:52.158454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e367e391e68a477d861a515f846a8dfa)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:52.160870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:52.393377 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e367e391e68a477d861a515f846a8dfa'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:52.394063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e367e391e68a477d861a515f846a8dfa'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:52.394964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e367e391e68a477d861a515f846a8dfa'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:52.413539 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e367e391e68a477d861a515f846a8dfa'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:52.555733 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e367e391e68a477d861a515f846a8dfa'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:52.571967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1211/2438] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:52 2022] DELETE /identity/v3/users/e367e391e68a477d861a515f846a8dfa => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:52.953870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:52.981245 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a91f5-3a69-4ca9-bc12-ad156b683079 service placement] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:53.070207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:53.071142 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:53.071421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:53.071724 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] PATH_INFO: `/v3/projects/c100074063c44d96adf93e0b0f080527` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:53.075115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c100074063c44d96adf93e0b0f080527)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:53.081238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:53.194637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c100074063c44d96adf93e0b0f080527', 'request_id': 'req-8c2e43c7-6fbb-4939-9273-99ff33d140dc'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:53.209910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c100074063c44d96adf93e0b0f080527', 'request_id': 'req-8c2e43c7-6fbb-4939-9273-99ff33d140dc'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:53.587821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:53.587821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:53.587821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:53.587821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074476af-e048-41f0-ae2f-2e38d849d04f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:53.590063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:53.590063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:53.590063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:53.590063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] PATH_INFO: `/v3/users/df400aee39a3498d8a26d55180075ab6` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:53.590576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df400aee39a3498d8a26d55180075ab6)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:53.590576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:53.733129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c100074063c44d96adf93e0b0f080527', 'request_id': 'req-8c2e43c7-6fbb-4939-9273-99ff33d140dc'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:53.747043 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] The token cache is being invalidate because project c100074063c44d96adf93e0b0f080527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:53.747334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:53.748000 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-8c2e43c7-6fbb-4939-9273-99ff33d140dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:53.749587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1228/2439] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:52 2022] DELETE /identity/v3/projects/c100074063c44d96adf93e0b0f080527 => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:53.754483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed300ba-71bd-439b-9712-b1065cf71eeb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:53.754772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed300ba-71bd-439b-9712-b1065cf71eeb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:53.755051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed300ba-71bd-439b-9712-b1065cf71eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:53.774119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4ed300ba-71bd-439b-9712-b1065cf71eeb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:53.775362 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4ed300ba-71bd-439b-9712-b1065cf71eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:53.784665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1229/2440] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:53.792683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2fa3c-0246-4729-ae6e-cb0156dc9130 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:53.792974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2fa3c-0246-4729-ae6e-cb0156dc9130 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:53.793248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2fa3c-0246-4729-ae6e-cb0156dc9130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:53.811645 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-11b2fa3c-0246-4729-ae6e-cb0156dc9130 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:53.812891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-11b2fa3c-0246-4729-ae6e-cb0156dc9130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:53.827252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1230/2441] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:53.828937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2fa3c-0246-4729-ae6e-cb0156dc9130 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:53.889688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2fa3c-0246-4729-ae6e-cb0156dc9130 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:53.889688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:53.889688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:53.889688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:53.890161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:53.890161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:03:53.890161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:53.890161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:53.890161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1231/2442] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:03:53.893294 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:53.904470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d39b-77dc-4707-8ad8-6488955aea10 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:53.909393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:53.910061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:53.910375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:53.910658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:53.911352 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:53.913308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:54.383431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1232/2443] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:53 2022] POST /identity/v3/projects => generated 367 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:54.388157 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:54.421150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df400aee39a3498d8a26d55180075ab6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:54.421943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df400aee39a3498d8a26d55180075ab6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:54.422597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df400aee39a3498d8a26d55180075ab6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:54.429233 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:54.435973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df400aee39a3498d8a26d55180075ab6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:54.451478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0d739-3c9d-4b90-902d-661e54abf956 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:54.456004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:54.456672 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:54.456980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:54.457258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:54.457948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:54.459920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:54.724676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df400aee39a3498d8a26d55180075ab6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:54.735697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1233/2444] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:54 2022] POST /identity/v3/users => generated 284 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:54.736806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:54.739385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:54.751161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1212/2445] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:03:53 2022] DELETE /identity/v3/users/df400aee39a3498d8a26d55180075ab6 => generated 0 bytes in 1275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:03:54.756346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c2349-41df-4b44-a66f-76029741b5f1 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:54.825040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:54.826069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:54.826503 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:54.826881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:54.827820 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:54.829800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:54.843942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1234/2446] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:54.845703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:54.858834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa937a-814b-4abb-96ee-5b74a8a0a5a9 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:54.866347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f76e6-bd0f-4095-937a-d4ec20d99b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:54.867061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f76e6-bd0f-4095-937a-d4ec20d99b79 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:54.867417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f76e6-bd0f-4095-937a-d4ec20d99b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:54.867758 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f76e6-bd0f-4095-937a-d4ec20d99b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:54.868506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f76e6-bd0f-4095-937a-d4ec20d99b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:54.870942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f76e6-bd0f-4095-937a-d4ec20d99b79 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:54.886258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1213/2447] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:54.887708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:54.902230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76571170-fec9-4934-b256-0bcd5e2991fc admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:54.905561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:54.906545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:54.906833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:54.907112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] PATH_INFO: `/v3/projects/8c570b0a8242449881cedecf5fce69ee/users/2fc90fda750b4ae6bce2b3b174582b49/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:54.924315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c570b0a8242449881cedecf5fce69ee, user_id=2fc90fda750b4ae6bce2b3b174582b49, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:54.926289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:55.079866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1235/2448] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:54 2022] PUT /identity/v3/projects/8c570b0a8242449881cedecf5fce69ee/users/2fc90fda750b4ae6bce2b3b174582b49/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:55.090532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.090882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.091163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.117951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] MFA Rules not processed for user `2fc90fda750b4ae6bce2b3b174582b49`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:55.119195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:55.183279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:55.185305 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:55.186000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:55.190224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1214/2449] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:55.201824 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:55.203405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:55.265684 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d56ce-7552-49b4-b24a-80801f49ad0f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:55.269829 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:55.270774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.271057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.271367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] PATH_INFO: `/v3/projects/3ae6c02cace9481d9b87e8f9d61323c3` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.271867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ae0be-a44f-47be-9f98-f83c8b1f1013 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:55.272499 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae6c02cace9481d9b87e8f9d61323c3)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:55.274644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:55.276818 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e583ef-97bc-4d50-ac4c-543b6a9b717c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:55.277483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e583ef-97bc-4d50-ac4c-543b6a9b717c service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.277776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e583ef-97bc-4d50-ac4c-543b6a9b717c service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.278050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e583ef-97bc-4d50-ac4c-543b6a9b717c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.288494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e583ef-97bc-4d50-ac4c-543b6a9b717c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:55.290200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e583ef-97bc-4d50-ac4c-543b6a9b717c service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:55.300381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1215/2450] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:03:55.445103 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:55.451603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae6c02cace9481d9b87e8f9d61323c3', 'request_id': 'req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:55.457598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae6c02cace9481d9b87e8f9d61323c3', 'request_id': 'req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:55.603485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae6c02cace9481d9b87e8f9d61323c3', 'request_id': 'req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:55.614496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] The token cache is being invalidate because project 3ae6c02cace9481d9b87e8f9d61323c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:03:55.632811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:55.632811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d8e2d1e1-6b2c-4013-b51e-c563b879a37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:03:55.632811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1236/2451] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:03:55 2022] DELETE /identity/v3/projects/3ae6c02cace9481d9b87e8f9d61323c3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:55.633113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726517f0-0ac4-4697-a40d-68d3e4069c17 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.633113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726517f0-0ac4-4697-a40d-68d3e4069c17 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.633113 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726517f0-0ac4-4697-a40d-68d3e4069c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.640713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-726517f0-0ac4-4697-a40d-68d3e4069c17 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:55.641898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-726517f0-0ac4-4697-a40d-68d3e4069c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:55.702271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1216/2452] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:55.710306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64669f-e340-42ec-94e5-a74a5575bfe7 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.710598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64669f-e340-42ec-94e5-a74a5575bfe7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.710880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64669f-e340-42ec-94e5-a74a5575bfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.730402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3b64669f-e340-42ec-94e5-a74a5575bfe7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:55.732133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3b64669f-e340-42ec-94e5-a74a5575bfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:55.741538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1237/2453] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:55.748542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726517f0-0ac4-4697-a40d-68d3e4069c17 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:55.758656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726517f0-0ac4-4697-a40d-68d3e4069c17 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:55.764115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:55.764775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.765058 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.765444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.766204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:03:55.766514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:55.768134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:55.779788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1217/2454] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:03:55.783974 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64669f-e340-42ec-94e5-a74a5575bfe7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:55.794332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64669f-e340-42ec-94e5-a74a5575bfe7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:55.799273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fae903-60a8-4386-9a12-f5b3e8448003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:55.799985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fae903-60a8-4386-9a12-f5b3e8448003 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.800313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fae903-60a8-4386-9a12-f5b3e8448003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.810155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fae903-60a8-4386-9a12-f5b3e8448003 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.810155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fae903-60a8-4386-9a12-f5b3e8448003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:55.810155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fae903-60a8-4386-9a12-f5b3e8448003 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:55.955883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1238/2455] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:55 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:03:55.962481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:55.963336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f23c3-c3aa-4a3e-a0bf-684f38f3e0f2 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:55.963674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f23c3-c3aa-4a3e-a0bf-684f38f3e0f2 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:55.963920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f23c3-c3aa-4a3e-a0bf-684f38f3e0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:55.973308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c0db5-4a2c-4b0e-982f-da215db954d7 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:55.982234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ef4f23c3-c3aa-4a3e-a0bf-684f38f3e0f2 None None] MFA Rules not processed for user `833efc71510e4bf8a5418671026a1b29`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:55.983459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ef4f23c3-c3aa-4a3e-a0bf-684f38f3e0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:56.034144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:56.034851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:56.035135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:56.035417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:56.036127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:56.037601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1239/2456] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:03:56.038063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:56.152911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1218/2457] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:55 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:56.157062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f23c3-c3aa-4a3e-a0bf-684f38f3e0f2 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:56.169111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f23c3-c3aa-4a3e-a0bf-684f38f3e0f2 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:56.174003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:56.174712 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:56.174998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:56.175284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:56.176015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:56.177638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:56.189332 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1240/2458] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:56.193458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:56.204315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabbc5-91c6-4879-a573-39bf3fda1d01 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:56.209613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34822f4-5472-412a-9efd-2b1187312147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:56.210341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34822f4-5472-412a-9efd-2b1187312147 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:56.210627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34822f4-5472-412a-9efd-2b1187312147 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:56.210948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34822f4-5472-412a-9efd-2b1187312147 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:56.211677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34822f4-5472-412a-9efd-2b1187312147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:56.213307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34822f4-5472-412a-9efd-2b1187312147 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:56.225473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1219/2459] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:56.231560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:56.241525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea9866-5528-4a93-bbbd-250fa0ee3e0b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:56.247273 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:56.248175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:56.248602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:56.248738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] PATH_INFO: `/v3/projects/aa3f5b360f02494cbc1830f9af91a54a/users/38bb6e59dfca4778b4d3dcf82b6ac9a0/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:56.268600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3f5b360f02494cbc1830f9af91a54a, user_id=38bb6e59dfca4778b4d3dcf82b6ac9a0, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:56.269494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:56.515079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1241/2460] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:56 2022] PUT /identity/v3/projects/aa3f5b360f02494cbc1830f9af91a54a/users/38bb6e59dfca4778b4d3dcf82b6ac9a0/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:56.539969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d878b-02fc-4ec3-a0ea-cef6b946ffa5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:56.539969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d878b-02fc-4ec3-a0ea-cef6b946ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:56.539969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d878b-02fc-4ec3-a0ea-cef6b946ffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:56.552619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:56.562904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b04d878b-02fc-4ec3-a0ea-cef6b946ffa5 None None] MFA Rules not processed for user `38bb6e59dfca4778b4d3dcf82b6ac9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:56.564151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b04d878b-02fc-4ec3-a0ea-cef6b946ffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:56.627894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1220/2461] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:56.632608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:56.693010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0659265-7c13-4e1d-8e83-21a1b069256d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:56.697630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:56.698324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:56.698629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:56.698905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:56.699592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:56.701508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:56.888496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1242/2462] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:03:56 2022] POST /identity/v3/projects => generated 371 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:03:56.901765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d878b-02fc-4ec3-a0ea-cef6b946ffa5 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:56.912784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d878b-02fc-4ec3-a0ea-cef6b946ffa5 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:56.981379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:56.982055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:56.982387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:56.982725 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:56.983360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:56.985295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:57.278060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1221/2463] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:03:56 2022] POST /identity/v3/users => generated 292 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:03:57.282315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:57.292639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc1faf-c04d-46e8-a279-24ef78e320b3 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:57.297154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:57.297853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:57.298129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:57.298469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:57.299199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:57.300868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:57.313586 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1243/2464] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:57.317665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:57.328266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719abbdc-d162-46d6-874d-c714230ce71c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:57.332785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:57.333670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:57.333955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:57.334287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] PATH_INFO: `/v3/projects/a968ec6782c5437dbd6cab0b3b8a7244/users/b76593f09bf447528dca762969927fa3/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:57.350850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a968ec6782c5437dbd6cab0b3b8a7244, user_id=b76593f09bf447528dca762969927fa3, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:57.353707 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:57.398869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:03:57.513696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1222/2465] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:57 2022] PUT /identity/v3/projects/a968ec6782c5437dbd6cab0b3b8a7244/users/b76593f09bf447528dca762969927fa3/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:57.517615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:57.530796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c739fe6-fd11-4aa5-9298-cc3a93d7f9cb admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:57.594683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90107a4-b112-4db0-9276-801c3ba69189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:57.594683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90107a4-b112-4db0-9276-801c3ba69189 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:57.594683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90107a4-b112-4db0-9276-801c3ba69189 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:57.594683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90107a4-b112-4db0-9276-801c3ba69189 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:57.595205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90107a4-b112-4db0-9276-801c3ba69189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:57.595205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90107a4-b112-4db0-9276-801c3ba69189 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:57.603928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1244/2466] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:03:57.608139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:57.618552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2bb86-8269-4084-b7ac-9c078cd4dc47 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:57.623438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:57.624439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:57.624720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:57.624992 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] PATH_INFO: `/v3/projects/a968ec6782c5437dbd6cab0b3b8a7244/users/b76593f09bf447528dca762969927fa3/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:57.626977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a968ec6782c5437dbd6cab0b3b8a7244, user_id=b76593f09bf447528dca762969927fa3, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:57.629656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:57.641551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1223/2467] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:03:57 2022] PUT /identity/v3/projects/a968ec6782c5437dbd6cab0b3b8a7244/users/b76593f09bf447528dca762969927fa3/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:03:57.654632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488dc5-4212-4873-acb9-95d76fddb3a3 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:57.654927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488dc5-4212-4873-acb9-95d76fddb3a3 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:57.655208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488dc5-4212-4873-acb9-95d76fddb3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:57.681162 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-03488dc5-4212-4873-acb9-95d76fddb3a3 None None] MFA Rules not processed for user `b76593f09bf447528dca762969927fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:03:57.682854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-03488dc5-4212-4873-acb9-95d76fddb3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:57.751622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1245/2468] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:03:57.765418 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:03:57.767171 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:03:57.824787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f438-495a-46ca-a52c-5685b8a6c81a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:03:57.828530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:03:57.829174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:03:57.829454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:03:57.829729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:03:57.839247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:03:57.840710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:03:57.850283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1224/2469] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:03:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:03:58.478409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:00.033114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03488dc5-4212-4873-acb9-95d76fddb3a3 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:00.049976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:00.051389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:00.109154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03488dc5-4212-4873-acb9-95d76fddb3a3 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:00.114567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:00.115492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:00.115766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:00.116098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] PATH_INFO: `/v3/users/38bb6e59dfca4778b4d3dcf82b6ac9a0` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:00.117469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38bb6e59dfca4778b4d3dcf82b6ac9a0)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:00.120258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:00.128007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:00.143639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8aa5-1c86-4b3a-8b64-0437f8f364b2 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:00.149500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad15730-d5ad-4dd4-9423-7fe0ae484f82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:00.149913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad15730-d5ad-4dd4-9423-7fe0ae484f82 service placement] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:00.150290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad15730-d5ad-4dd4-9423-7fe0ae484f82 service placement] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:00.151126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad15730-d5ad-4dd4-9423-7fe0ae484f82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:00.153161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fad15730-d5ad-4dd4-9423-7fe0ae484f82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:00.165499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad15730-d5ad-4dd4-9423-7fe0ae484f82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:00.167723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad15730-d5ad-4dd4-9423-7fe0ae484f82 service placement] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:00.177900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1225/2470] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:00.338874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38bb6e59dfca4778b4d3dcf82b6ac9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:00.339729 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38bb6e59dfca4778b4d3dcf82b6ac9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:00.347997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38bb6e59dfca4778b4d3dcf82b6ac9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:00.444370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38bb6e59dfca4778b4d3dcf82b6ac9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:00.457367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38bb6e59dfca4778b4d3dcf82b6ac9a0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:00.464272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1246/2471] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:00 2022] DELETE /identity/v3/users/38bb6e59dfca4778b4d3dcf82b6ac9a0 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:00.468825 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:00.469119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:00.469665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:00.487696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:00.489797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:00.559074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1226/2472] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:00.569338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:00.574140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:00.636073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12c847-c8b6-4dc6-bd07-713265dbc3c8 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:00.641023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:00.641679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:00.641964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:00.642346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:00.652743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:00.654779 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:00.664646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1247/2473] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:01.120307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:01.122400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:01.137322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cc251-4094-4b99-978a-2889115d30d9 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:01.142904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:01.143962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:01.144297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:01.144636 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] PATH_INFO: `/v3/projects/aa3f5b360f02494cbc1830f9af91a54a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:01.145977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3f5b360f02494cbc1830f9af91a54a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:01.147945 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:01.287674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3f5b360f02494cbc1830f9af91a54a', 'request_id': 'req-c5f1fe6d-9b41-4106-80ac-378871db748d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:01.390811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3f5b360f02494cbc1830f9af91a54a', 'request_id': 'req-c5f1fe6d-9b41-4106-80ac-378871db748d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:01.401131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3f5b360f02494cbc1830f9af91a54a', 'request_id': 'req-c5f1fe6d-9b41-4106-80ac-378871db748d'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:01.420552 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] The token cache is being invalidate because project aa3f5b360f02494cbc1830f9af91a54a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:01.421030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:01.421839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-c5f1fe6d-9b41-4106-80ac-378871db748d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:01.423865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1227/2474] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:01 2022] DELETE /identity/v3/projects/aa3f5b360f02494cbc1830f9af91a54a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:01.427878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:01.440619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832c0cf-c133-4448-b4ba-3b8faa1ac61b service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:01.514216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:01.515402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:01.515687 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:01.516171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] PATH_INFO: `/v3/users/b76593f09bf447528dca762969927fa3` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:01.518005 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b76593f09bf447528dca762969927fa3)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:01.520966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:01.822262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:01.837302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b76593f09bf447528dca762969927fa3'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:01.838427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b76593f09bf447528dca762969927fa3'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:01.847218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b76593f09bf447528dca762969927fa3'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:02.010405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b76593f09bf447528dca762969927fa3'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:02.031569 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b76593f09bf447528dca762969927fa3'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:02.038216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1248/2475] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:01 2022] DELETE /identity/v3/users/b76593f09bf447528dca762969927fa3 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:02.184599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:02.185104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:02.185210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:02.199455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:02.206194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:02.207487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:02.277266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1228/2476] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:02.299701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:02.301472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:02.398084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c5c4e-2050-4407-a82a-d0cd722faa04 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:02.415733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41257-f745-464e-8f61-4fda8804b616 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:02.417977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41257-f745-464e-8f61-4fda8804b616 service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:02.418337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41257-f745-464e-8f61-4fda8804b616 service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:02.418738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41257-f745-464e-8f61-4fda8804b616 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:02.432389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41257-f745-464e-8f61-4fda8804b616 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:02.435186 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41257-f745-464e-8f61-4fda8804b616 service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:02.492137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1249/2477] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:04:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:04:02.665286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:02.666968 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:02.858984 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9713256-5888-4762-88f7-58b61a9b4668 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:02.859585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:02.860660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:02.861810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:02.862285 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] PATH_INFO: `/v3/projects/a968ec6782c5437dbd6cab0b3b8a7244` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:02.863722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a968ec6782c5437dbd6cab0b3b8a7244)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:02.865987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:02.991973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a968ec6782c5437dbd6cab0b3b8a7244', 'request_id': 'req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:03.124923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a968ec6782c5437dbd6cab0b3b8a7244', 'request_id': 'req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:03.134484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a968ec6782c5437dbd6cab0b3b8a7244', 'request_id': 'req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:03.161098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] The token cache is being invalidate because project a968ec6782c5437dbd6cab0b3b8a7244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:03.161515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:03.163594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-13d0ffb0-3e8e-4dc3-b1c0-f0322a1b1407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:03.166779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1229/2478] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:02 2022] DELETE /identity/v3/projects/a968ec6782c5437dbd6cab0b3b8a7244 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:03.172035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9267-8e8a-45ce-81ea-7ff4a22394d1 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:03.172035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9267-8e8a-45ce-81ea-7ff4a22394d1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:03.177934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9267-8e8a-45ce-81ea-7ff4a22394d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:03.194512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5b5a9267-8e8a-45ce-81ea-7ff4a22394d1 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:03.195939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5b5a9267-8e8a-45ce-81ea-7ff4a22394d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:03.288040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1250/2479] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:03.297974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084d78c-1a0f-4276-95e1-d6d9863bee30 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:03.298735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084d78c-1a0f-4276-95e1-d6d9863bee30 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:03.298735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084d78c-1a0f-4276-95e1-d6d9863bee30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:03.340759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7084d78c-1a0f-4276-95e1-d6d9863bee30 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:03.344505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7084d78c-1a0f-4276-95e1-d6d9863bee30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:03.404297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1230/2480] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:03.408518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9267-8e8a-45ce-81ea-7ff4a22394d1 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:03.433931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9267-8e8a-45ce-81ea-7ff4a22394d1 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:03.453033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:03.454072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:03.454631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:03.457012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:03.457012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:03.457012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:03.458313 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:03.475572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1251/2481] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:04:03.480633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084d78c-1a0f-4276-95e1-d6d9863bee30 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:03.491703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084d78c-1a0f-4276-95e1-d6d9863bee30 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:03.497749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:03.498797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:03.499281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:03.499736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:03.500689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:03.503169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:03.646411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1231/2482] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:03 2022] POST /identity/v3/projects => generated 357 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:03.651261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:03.670635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802d6a-ab8a-45cc-957a-cb9328807e37 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:03.698543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:03.765995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:03.767996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:03.768615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:03.769107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:03.771116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:03.800211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:03.947610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1252/2483] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:03 2022] POST /identity/v3/users => generated 279 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:03.952866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:03.976174 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367523-4333-4e37-9e7c-9dd8bd384c71 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:03.984627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:03.986056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:03.986692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:03.987150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:03.988309 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:04.002287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:04.034802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1232/2484] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:04.036272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:04.049292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4123796-3d12-49a5-b967-9216530cef3b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:04.055811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7169c3e-8eee-41f4-aa3f-104ec2ff75a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:04.057172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169c3e-8eee-41f4-aa3f-104ec2ff75a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:04.058801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169c3e-8eee-41f4-aa3f-104ec2ff75a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:04.058801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169c3e-8eee-41f4-aa3f-104ec2ff75a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:04.058801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7169c3e-8eee-41f4-aa3f-104ec2ff75a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:04.060793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7169c3e-8eee-41f4-aa3f-104ec2ff75a3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:04.075715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1253/2485] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:04.080635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:04.086496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:04.095409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fdd62-107b-4cfd-853b-5d2b3509b872 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:04.106601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:04.107932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:04.108354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:04.108649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] PATH_INFO: `/v3/projects/5c0de621f4984903ad98d3fa3f94d373/users/3ab90db5851f4182beac27743502e863/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:04.145335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0de621f4984903ad98d3fa3f94d373, user_id=3ab90db5851f4182beac27743502e863, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:04.156566 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:04.251489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1233/2486] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:04 2022] PUT /identity/v3/projects/5c0de621f4984903ad98d3fa3f94d373/users/3ab90db5851f4182beac27743502e863/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:04.262897 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32af10-7435-4d11-b02f-9ff2cf3e599a None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:04.263196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32af10-7435-4d11-b02f-9ff2cf3e599a None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:04.263479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32af10-7435-4d11-b02f-9ff2cf3e599a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:04.303548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5d32af10-7435-4d11-b02f-9ff2cf3e599a None None] MFA Rules not processed for user `3ab90db5851f4182beac27743502e863`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:04.308451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5d32af10-7435-4d11-b02f-9ff2cf3e599a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:04.383942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1254/2487] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:04.389711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:04.481749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85873318-0240-470f-9ab7-2a1c2dc21efe admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:04.488126 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:04.489224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:04.489696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:04.490538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:04.491747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:04.494618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:04.598845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32af10-7435-4d11-b02f-9ff2cf3e599a None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:04.609007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1234/2488] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:04 2022] POST /identity/v3/projects => generated 357 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:04.611783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32af10-7435-4d11-b02f-9ff2cf3e599a None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:04.676344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:04.677032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:04.677301 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:04.677576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:04.678299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:04.680462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:04.811011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1255/2489] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:04 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:04.815234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:04.826528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824382cb-c6a8-4c68-8ccf-3a7ebbf00eda admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:04.831206 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:04.831874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:04.832159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:04.832435 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:04.833177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:04.834865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:04.847253 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1235/2490] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:04.851380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:04.862440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52610d-6448-4fe3-a448-d137b0775360 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:04.869598 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:04.871442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:04.875069 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:04.875378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] PATH_INFO: `/v3/projects/53b39fc114ee4202ab8db00474e4c7ed/users/14ccd639f9b840e799c4c8eb11859c3a/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:04.903230 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b39fc114ee4202ab8db00474e4c7ed, user_id=14ccd639f9b840e799c4c8eb11859c3a, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:04.907115 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:05.005560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:05.018240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1256/2491] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:04 2022] PUT /identity/v3/projects/53b39fc114ee4202ab8db00474e4c7ed/users/14ccd639f9b840e799c4c8eb11859c3a/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:05.020380 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:05.033190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258661a-50a7-4fb8-a52b-987ecb8c8719 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:05.113220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86399ec-8ab8-4eb9-8201-951600ce43aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:05.117932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86399ec-8ab8-4eb9-8201-951600ce43aa admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:05.117932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86399ec-8ab8-4eb9-8201-951600ce43aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:05.117932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86399ec-8ab8-4eb9-8201-951600ce43aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:05.117932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86399ec-8ab8-4eb9-8201-951600ce43aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:05.117932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86399ec-8ab8-4eb9-8201-951600ce43aa admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:05.130288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1236/2492] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:05.135869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:05.168858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d518ae9-c2f4-440c-93db-2c6a32b6f71c admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:05.175215 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:05.176222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:05.176575 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:05.176920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] PATH_INFO: `/v3/projects/53b39fc114ee4202ab8db00474e4c7ed/users/14ccd639f9b840e799c4c8eb11859c3a/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:05.195198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b39fc114ee4202ab8db00474e4c7ed, user_id=14ccd639f9b840e799c4c8eb11859c3a, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:05.199254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:05.231150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1257/2493] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:05 2022] PUT /identity/v3/projects/53b39fc114ee4202ab8db00474e4c7ed/users/14ccd639f9b840e799c4c8eb11859c3a/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:05.248202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646fce2-84be-43ae-8b9d-a36540c73067 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:05.248500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646fce2-84be-43ae-8b9d-a36540c73067 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:05.248764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646fce2-84be-43ae-8b9d-a36540c73067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:05.280602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d646fce2-84be-43ae-8b9d-a36540c73067 None None] MFA Rules not processed for user `14ccd639f9b840e799c4c8eb11859c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:05.281879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d646fce2-84be-43ae-8b9d-a36540c73067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:05.325959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:05.360881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1237/2494] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:05 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:05.373291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:05.375348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:05.458849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b588d-2473-4b94-8ffb-612f3207bc94 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:05.463265 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:05.464003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:05.464343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:05.464631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:05.479966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:05.481234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:05.493471 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1258/2495] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:05.818297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646fce2-84be-43ae-8b9d-a36540c73067 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:05.834339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646fce2-84be-43ae-8b9d-a36540c73067 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:05.842314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fe03f-7e72-49eb-ad9b-84d0fbb46d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:05.843662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe03f-7e72-49eb-ad9b-84d0fbb46d62 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:05.844125 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe03f-7e72-49eb-ad9b-84d0fbb46d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:05.844781 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe03f-7e72-49eb-ad9b-84d0fbb46d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:05.918522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fe03f-7e72-49eb-ad9b-84d0fbb46d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:05.920329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fe03f-7e72-49eb-ad9b-84d0fbb46d62 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:05.932815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1238/2496] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:07.954576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:07.956087 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:08.030237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5cf55-8249-4c65-a9fe-a0b4d0b11d3f service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.035083 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.036078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.036430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.036780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] PATH_INFO: `/v3/users/2fc90fda750b4ae6bce2b3b174582b49` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.038056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc90fda750b4ae6bce2b3b174582b49)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.040102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.262070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc90fda750b4ae6bce2b3b174582b49'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.262802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc90fda750b4ae6bce2b3b174582b49'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.269827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc90fda750b4ae6bce2b3b174582b49'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.335421 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc90fda750b4ae6bce2b3b174582b49'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.351992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc90fda750b4ae6bce2b3b174582b49'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:08.359749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1259/2497] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:07 2022] DELETE /identity/v3/users/2fc90fda750b4ae6bce2b3b174582b49 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:08.364224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.364660 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.364873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.385778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:08.387866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:08.501587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1239/2498] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:08.513866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:08.515981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:08.530049 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:08.607193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c0e07-a66b-452d-a0a3-177c4dc59d75 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:08.611200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343fc99-817d-4e0f-ae0e-d26454f6b93b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:08.611969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343fc99-817d-4e0f-ae0e-d26454f6b93b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:08.612258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343fc99-817d-4e0f-ae0e-d26454f6b93b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:08.612538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343fc99-817d-4e0f-ae0e-d26454f6b93b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:08.622938 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343fc99-817d-4e0f-ae0e-d26454f6b93b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:08.626752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343fc99-817d-4e0f-ae0e-d26454f6b93b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:08.634771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1260/2499] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:09.211505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:09.213767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:09.241600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba70bf-5b3f-47ee-9a11-f579716c5a27 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:09.248245 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:09.249277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:09.249670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:09.250011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] PATH_INFO: `/v3/projects/8c570b0a8242449881cedecf5fce69ee` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:09.251881 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c570b0a8242449881cedecf5fce69ee)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:09.253903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:09.504143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c570b0a8242449881cedecf5fce69ee', 'request_id': 'req-769d4153-6aa0-4849-8608-759945445441'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:09.700389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c570b0a8242449881cedecf5fce69ee', 'request_id': 'req-769d4153-6aa0-4849-8608-759945445441'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:09.709336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c570b0a8242449881cedecf5fce69ee', 'request_id': 'req-769d4153-6aa0-4849-8608-759945445441'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:09.713970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:09.740928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] The token cache is being invalidate because project 8c570b0a8242449881cedecf5fce69ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:09.741229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:09.743716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-769d4153-6aa0-4849-8608-759945445441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:09.744246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1240/2500] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:09 2022] DELETE /identity/v3/projects/8c570b0a8242449881cedecf5fce69ee => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:09.749250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004686-138d-4878-ad6f-b5d49469bab7 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:09.750111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004686-138d-4878-ad6f-b5d49469bab7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:09.750763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13004686-138d-4878-ad6f-b5d49469bab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:09.784258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-13004686-138d-4878-ad6f-b5d49469bab7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:09.855386 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-13004686-138d-4878-ad6f-b5d49469bab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:09.864706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1261/2501] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:09.879696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a95c2-9308-4457-90bf-731573170cce None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:09.879696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a95c2-9308-4457-90bf-731573170cce None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:09.880205 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a95c2-9308-4457-90bf-731573170cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:09.904544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1e6a95c2-9308-4457-90bf-731573170cce None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:09.908732 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1e6a95c2-9308-4457-90bf-731573170cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:09.918762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1241/2502] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:09.923198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13004686-138d-4878-ad6f-b5d49469bab7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:09.935097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13004686-138d-4878-ad6f-b5d49469bab7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:09.940563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:09.941367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:09.941805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:09.942257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:09.943190 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:09.943635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:09.945783 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:09.961482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1262/2503] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:04:09.967169 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a95c2-9308-4457-90bf-731573170cce None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:09.977462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a95c2-9308-4457-90bf-731573170cce None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:09.983833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:09.986251 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:09.992924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:09.992924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:09.992924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:09.992924 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:10.112437 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1242/2504] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:09 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:10.118031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:10.130370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2eab-c629-424d-9fc3-85eb8cc4f35b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:10.215570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:10.238327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.238327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.238327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.238327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:10.238327 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:10.364651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1263/2505] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:10 2022] POST /identity/v3/users => generated 290 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:10.369354 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:10.380244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfad375-e6f4-4024-875f-02854b991c11 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:10.385642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:10.386360 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.386646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.386938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.387704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:10.389804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:10.404997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1243/2506] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:10.413238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:10.424191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc31274-6f7a-4d7c-9d74-dcf1977bf396 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:10.428993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c81ff3-297d-4a34-9b64-ea332d7944aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:10.429669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c81ff3-297d-4a34-9b64-ea332d7944aa admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.429960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c81ff3-297d-4a34-9b64-ea332d7944aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.430264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c81ff3-297d-4a34-9b64-ea332d7944aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.431752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c81ff3-297d-4a34-9b64-ea332d7944aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:10.433614 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c81ff3-297d-4a34-9b64-ea332d7944aa admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:10.448657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1264/2507] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:10.455434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:10.466676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d46bec-5ea1-4110-8af0-986fcc1ab4e6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:10.472958 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:10.473867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.474208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.474486 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] PATH_INFO: `/v3/projects/f5bbf4c2727a422da32fc532c1bf3708/users/86e8f1d48d9942ca813ba1cf82406c8a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.492597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5bbf4c2727a422da32fc532c1bf3708, user_id=86e8f1d48d9942ca813ba1cf82406c8a, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:10.496494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:10.651007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1244/2508] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:10 2022] PUT /identity/v3/projects/f5bbf4c2727a422da32fc532c1bf3708/users/86e8f1d48d9942ca813ba1cf82406c8a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:10.662246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabf42f-15a3-401a-9b1e-c22e3d40e14c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.662568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabf42f-15a3-401a-9b1e-c22e3d40e14c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.662826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabf42f-15a3-401a-9b1e-c22e3d40e14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.701367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8dabf42f-15a3-401a-9b1e-c22e3d40e14c None None] MFA Rules not processed for user `86e8f1d48d9942ca813ba1cf82406c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:10.702631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8dabf42f-15a3-401a-9b1e-c22e3d40e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:10.706823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:10.773070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1265/2509] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:10.777746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:10.881839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce236ecc-b623-4579-b0b9-9cf44b534af6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:10.889557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:10.890614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:10.890915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:10.892943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:10.894329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:10.895787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:11.043123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1245/2510] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:10 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:11.044453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabf42f-15a3-401a-9b1e-c22e3d40e14c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:11.055367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabf42f-15a3-401a-9b1e-c22e3d40e14c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:11.115677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:11.116357 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:11.116646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:11.116933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:11.117626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:11.123642 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:11.243473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1266/2511] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:11 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:11.245894 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:11.259380 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741efa9-a87d-4def-95c4-8292df3418e6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:11.265787 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:11.266505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:11.266842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:11.267315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:11.268231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:11.269871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:11.282034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1246/2512] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:11.286156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:11.296686 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee047-87e0-4757-8b97-493fe9b5514e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:11.301328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:11.302272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:11.302560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:11.302836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] PATH_INFO: `/v3/projects/1666d14c50774afc9abc76002805e61d/users/0028fbe692764eb79ba6f5be17d24700/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:11.318665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1666d14c50774afc9abc76002805e61d, user_id=0028fbe692764eb79ba6f5be17d24700, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:11.321342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:11.456336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1267/2513] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:11 2022] PUT /identity/v3/projects/1666d14c50774afc9abc76002805e61d/users/0028fbe692764eb79ba6f5be17d24700/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:11.459160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:11.470271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba65246-612e-4ea8-8c53-c9bb36c614bc admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:11.528748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863df14b-dc40-4d02-8b2c-19944d8efba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:11.530004 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863df14b-dc40-4d02-8b2c-19944d8efba9 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:11.530492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863df14b-dc40-4d02-8b2c-19944d8efba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:11.530947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863df14b-dc40-4d02-8b2c-19944d8efba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:11.531686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863df14b-dc40-4d02-8b2c-19944d8efba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:11.533752 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863df14b-dc40-4d02-8b2c-19944d8efba9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:11.548208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1247/2514] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:11.549371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:11.560453 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c092f-84c3-4ddc-b397-f51371ff98d2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:11.565074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:11.565955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:11.566259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:11.566530 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] PATH_INFO: `/v3/projects/1666d14c50774afc9abc76002805e61d/users/0028fbe692764eb79ba6f5be17d24700/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:11.568484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1666d14c50774afc9abc76002805e61d, user_id=0028fbe692764eb79ba6f5be17d24700, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:11.571167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:11.583791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1268/2515] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:11 2022] PUT /identity/v3/projects/1666d14c50774afc9abc76002805e61d/users/0028fbe692764eb79ba6f5be17d24700/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:11.596368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:11.596837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:11.597283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:11.603735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:11.628424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] MFA Rules not processed for user `0028fbe692764eb79ba6f5be17d24700`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:11.629746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:11.722024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1248/2516] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:11.733740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:11.735540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:11.796857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8fdd9-dd4a-4706-aea7-45fb5becd76b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:11.802066 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:11.806908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:11.806908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:11.806908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:11.813397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:11.815018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:11.825243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1269/2517] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:11.935992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:12.086326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:12.087905 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:12.105164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd0f44-cf98-44dd-822b-5433a2a2bef6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:12.108879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:12.109529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:12.109808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:12.110081 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:12.168791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:12.170316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:12.179396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1249/2518] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:22.318124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=61 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:23.953196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:23.954661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:24.011975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27c199-65a4-40be-8404-5a17820e9cc0 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:24.015841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:24.016478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:24.016755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:24.017029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:24.019012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:24.083154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:24.084603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:24.095136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1270/2519] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:25.135235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:25.137290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:25.149630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f29cb-bf49-4c03-baf0-2cbcc55621df service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:25.155680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b896f-948b-4526-b917-9ecad33bd212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:25.155680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b896f-948b-4526-b917-9ecad33bd212 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:25.155680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b896f-948b-4526-b917-9ecad33bd212 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:25.155680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b896f-948b-4526-b917-9ecad33bd212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:25.157097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-648b896f-948b-4526-b917-9ecad33bd212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:25.221702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b896f-948b-4526-b917-9ecad33bd212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:25.223208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b896f-948b-4526-b917-9ecad33bd212 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:25.232993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1250/2520] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:27.821721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:27.837482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a13c0-53b8-45b6-912e-9f7e9d3dbd1b service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:27.843979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:27.845328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:27.845683 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:27.846113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] PATH_INFO: `/v3/users/86e8f1d48d9942ca813ba1cf82406c8a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:27.858968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86e8f1d48d9942ca813ba1cf82406c8a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:27.863512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:28.229112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86e8f1d48d9942ca813ba1cf82406c8a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:28.229854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86e8f1d48d9942ca813ba1cf82406c8a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:28.237061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86e8f1d48d9942ca813ba1cf82406c8a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:28.318015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86e8f1d48d9942ca813ba1cf82406c8a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:28.333590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86e8f1d48d9942ca813ba1cf82406c8a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:28.340462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1271/2521] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:27 2022] DELETE /identity/v3/users/86e8f1d48d9942ca813ba1cf82406c8a => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:28.345032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:28.345884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:28.346438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:28.364367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:28.365590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:28.442040 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1251/2522] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:28.443637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:28.445078 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:28.473840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:28.505934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4272bf3-e9e2-428e-80d0-c17905e2a58b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:28.509562 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:28.510246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:28.510541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:28.511474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:28.521116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:28.522641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:28.532961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1272/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:29.141011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:29.145791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:29.160402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee5341-ce52-4dae-8cea-1d953ee9710d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:29.165322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:29.166514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:29.167884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:29.168408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] PATH_INFO: `/v3/users/3ab90db5851f4182beac27743502e863` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:29.169593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab90db5851f4182beac27743502e863)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:29.171607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:29.246976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:29.248618 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:29.267719 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bd1a1-4fe2-46fd-83e2-25f511afa689 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:29.296064 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:29.297338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:29.297797 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:29.298322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] PATH_INFO: `/v3/projects/f5bbf4c2727a422da32fc532c1bf3708` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:29.302361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5bbf4c2727a422da32fc532c1bf3708)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:29.304764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:29.395674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab90db5851f4182beac27743502e863'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.396345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab90db5851f4182beac27743502e863'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.396951 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab90db5851f4182beac27743502e863'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.411068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab90db5851f4182beac27743502e863'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.428739 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5bbf4c2727a422da32fc532c1bf3708', 'request_id': 'req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.433795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5bbf4c2727a422da32fc532c1bf3708', 'request_id': 'req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.500639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-beac407a-8cee-4258-ad65-755a2d804cbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab90db5851f4182beac27743502e863'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.514624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1252/2524] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:29 2022] DELETE /identity/v3/users/3ab90db5851f4182beac27743502e863 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:29.518975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:29.519268 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:29.519548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:29.536849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:29.538124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:29.543402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:29.599313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1253/2525] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:29.608962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:29.610357 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:29.665213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f34c3-c0d1-4080-a9f0-2828f8e8727e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:29.669246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:29.669900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:29.670203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:29.670591 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:29.677804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5bbf4c2727a422da32fc532c1bf3708', 'request_id': 'req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.680854 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:29.682033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:29.699583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] The token cache is being invalidate because project f5bbf4c2727a422da32fc532c1bf3708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:29.699583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.699583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:29.699583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1273/2526] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:29 2022] DELETE /identity/v3/projects/f5bbf4c2727a422da32fc532c1bf3708 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:29.699917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1254/2527] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:29.700701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:29.700826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:29.701505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:29.701635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:29.715822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2586a8-21e7-41c0-8c23-e00fffb4fc36 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:29.721506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:29.722561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:29.722853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:29.723130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] PATH_INFO: `/v3/users/0028fbe692764eb79ba6f5be17d24700` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:29.724400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0028fbe692764eb79ba6f5be17d24700)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:29.726401 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:29.778395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b4e2d-1e8b-4f4c-ae72-87780e2268bf admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:29.782973 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:29.783854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:29.784290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:29.784573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] PATH_INFO: `/v3/users/bf1b5d8cbfbb450da8878719c4aa6712` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:29.785816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1b5d8cbfbb450da8878719c4aa6712)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:29.787795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:30.059600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1b5d8cbfbb450da8878719c4aa6712'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.060487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1b5d8cbfbb450da8878719c4aa6712'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.063183 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:30.069033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1b5d8cbfbb450da8878719c4aa6712'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.094428 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0028fbe692764eb79ba6f5be17d24700'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.095220 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0028fbe692764eb79ba6f5be17d24700'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.095929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0028fbe692764eb79ba6f5be17d24700'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.112074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0028fbe692764eb79ba6f5be17d24700'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.175458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1b5d8cbfbb450da8878719c4aa6712'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.197364 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1b5d8cbfbb450da8878719c4aa6712'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.203979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1274/2528] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:29 2022] DELETE /identity/v3/users/bf1b5d8cbfbb450da8878719c4aa6712 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:30.284104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0028fbe692764eb79ba6f5be17d24700'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.298603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1255/2529] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:29 2022] DELETE /identity/v3/users/0028fbe692764eb79ba6f5be17d24700 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:30.401155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:30.403602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:30.512743 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af638b1-1031-486c-aca9-89841a54ed25 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:30.576947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:30.577753 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:30.578041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:30.578351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] PATH_INFO: `/v3/projects/5c0de621f4984903ad98d3fa3f94d373` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:30.579534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0de621f4984903ad98d3fa3f94d373)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:30.581470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:30.606431 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:30.608056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:30.628426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0186e-7bae-4d41-9f3a-c0e69c69bf07 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:30.633962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:30.635042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:30.635421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:30.635974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] PATH_INFO: `/v3/projects/5f7ed2c773bd43c7a0845599bfb902ec` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:30.638196 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7ed2c773bd43c7a0845599bfb902ec)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:30.640623 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:30.815847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7ed2c773bd43c7a0845599bfb902ec', 'request_id': 'req-a42619d3-30f1-4f03-9046-3070fc240040'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.822513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0de621f4984903ad98d3fa3f94d373', 'request_id': 'req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.830012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0de621f4984903ad98d3fa3f94d373', 'request_id': 'req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.959115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:30.960302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7ed2c773bd43c7a0845599bfb902ec', 'request_id': 'req-a42619d3-30f1-4f03-9046-3070fc240040'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.968173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7ed2c773bd43c7a0845599bfb902ec', 'request_id': 'req-a42619d3-30f1-4f03-9046-3070fc240040'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.980101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] The token cache is being invalidate because project 5f7ed2c773bd43c7a0845599bfb902ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:30.980375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.981052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:30.982686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1256/2530] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:30 2022] DELETE /identity/v3/projects/5f7ed2c773bd43c7a0845599bfb902ec => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:30.987851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:30.987851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:31.050500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42619d3-30f1-4f03-9046-3070fc240040 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:31.059062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0de621f4984903ad98d3fa3f94d373', 'request_id': 'req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.076456 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] The token cache is being invalidate because project 5c0de621f4984903ad98d3fa3f94d373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:31.076745 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.077474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-3496f9fb-0699-4d93-819c-ab1e8a39fd5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.080167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1275/2531] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:30 2022] DELETE /identity/v3/projects/5c0de621f4984903ad98d3fa3f94d373 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:31.084160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.084516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.084882 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.127205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:31.127205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:31.127674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:31.127674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.127674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.127674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] PATH_INFO: `/v3/projects/1666d14c50774afc9abc76002805e61d` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.128042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1666d14c50774afc9abc76002805e61d)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:31.128042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:31.128214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1276/2532] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:31.128214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:31.128214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:31.136107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593010a-6912-4a97-9666-68deade3d4d1 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:31.141479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:31.142602 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.142892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.143168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] PATH_INFO: `/v3/users/14ccd639f9b840e799c4c8eb11859c3a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.144473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ccd639f9b840e799c4c8eb11859c3a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:31.146529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:31.397556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ccd639f9b840e799c4c8eb11859c3a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.398871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ccd639f9b840e799c4c8eb11859c3a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.399721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1666d14c50774afc9abc76002805e61d', 'request_id': 'req-db1d3f15-d90f-41dd-a452-799f4c48b8c7'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.409547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ccd639f9b840e799c4c8eb11859c3a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.482969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1666d14c50774afc9abc76002805e61d', 'request_id': 'req-db1d3f15-d90f-41dd-a452-799f4c48b8c7'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.489029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1666d14c50774afc9abc76002805e61d', 'request_id': 'req-db1d3f15-d90f-41dd-a452-799f4c48b8c7'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.501207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] The token cache is being invalidate because project 1666d14c50774afc9abc76002805e61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:31.501495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.502121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-db1d3f15-d90f-41dd-a452-799f4c48b8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.503843 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1257/2533] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:30 2022] DELETE /identity/v3/projects/1666d14c50774afc9abc76002805e61d => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:31.506201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8693ad-f9fb-4810-85a6-91f868c4e4c2 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.506476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8693ad-f9fb-4810-85a6-91f868c4e4c2 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.506751 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8693ad-f9fb-4810-85a6-91f868c4e4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.534847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-8a8693ad-f9fb-4810-85a6-91f868c4e4c2 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:31.536178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8a8693ad-f9fb-4810-85a6-91f868c4e4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:31.612813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1258/2534] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:31.614342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfa344-2b96-43d9-885e-d904feca8bb6 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.614679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfa344-2b96-43d9-885e-d904feca8bb6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.615100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfa344-2b96-43d9-885e-d904feca8bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.631979 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ccd639f9b840e799c4c8eb11859c3a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.636202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-a7cfa344-2b96-43d9-885e-d904feca8bb6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:31.637458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-a7cfa344-2b96-43d9-885e-d904feca8bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:31.646761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1259/2535] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:31.647619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfa344-2b96-43d9-885e-d904feca8bb6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:31.653721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f1d287ce-4f5a-40be-a850-28d80c523094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ccd639f9b840e799c4c8eb11859c3a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:31.657512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfa344-2b96-43d9-885e-d904feca8bb6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:31.660611 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1277/2536] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:31 2022] DELETE /identity/v3/users/14ccd639f9b840e799c4c8eb11859c3a => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:31.662068 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797ef3b-9109-4977-bf5e-622b48e24b4b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.662556 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797ef3b-9109-4977-bf5e-622b48e24b4b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.662833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797ef3b-9109-4977-bf5e-622b48e24b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.690845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:31.691641 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.696313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.696313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.696313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:31.696313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:31.696313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:31.696833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-f797ef3b-9109-4977-bf5e-622b48e24b4b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:31.698312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f797ef3b-9109-4977-bf5e-622b48e24b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:31.709160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1260/2537] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:04:31.713554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:31.726532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddada6e-f93b-4157-9865-724af13d61d6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:31.771336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1278/2538] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:31.777113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797ef3b-9109-4977-bf5e-622b48e24b4b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:31.789132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797ef3b-9109-4977-bf5e-622b48e24b4b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:31.796023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:31.796841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.797196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.797521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.798281 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:31.798878 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:31.800596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:31.809383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:31.810160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.810594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.811212 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.811572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:31.813605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:31.814833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1279/2539] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:04:31.818887 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:31.829074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8ac26-8c19-452f-bcc5-93093323875c admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:31.833800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:31.834516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:31.834798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:31.835206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:31.835978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:31.837868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:31.994310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1280/2540] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:31.994524 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1261/2541] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/projects => generated 365 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:31.996385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:31.996516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.070381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982c58-2d91-4dbf-92f1-9aad92a4bd89 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.072761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:32.076157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.077434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.077434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.077434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.078153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.079287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04eedb-8a79-4844-b7c3-a49ab496e97d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.080189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.084132 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.084843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.085129 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.085410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.086104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.088101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.168180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1262/2542] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/users => generated 283 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:32.172259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.181777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae7425-25fe-4579-87f2-55a099e5dde6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.187180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.187829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.188115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.188396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.189228 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.190875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.201604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1263/2543] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:32.205547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.215265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3393b-020b-4ffa-a3a6-326e028fbbaf admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.220302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.220927 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.221215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.221496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.222231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.223801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.236152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1281/2544] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:31 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:32.236339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1264/2545] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:32.237287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.239969 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.249284 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb6f0f-8f2b-47ad-8dad-2863e1c5d358 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.250559 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4369ea1-8e66-407a-a78f-07196615503d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.254142 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.254908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.255275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.255620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.255770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.256512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.256692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.257008 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.257288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] PATH_INFO: `/v3/projects/423a4b3ce644466fb21289009a186292/users/a193f92de13744a0a4b8cd4e9242f403/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.258402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.275440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1282/2546] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:32.278548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.279748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423a4b3ce644466fb21289009a186292, user_id=a193f92de13744a0a4b8cd4e9242f403, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.283263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.291138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:32.307290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021e978-51cf-4734-8428-e2b06c45eaf9 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.321239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.321239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.321239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.321239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] PATH_INFO: `/v3/projects/53b39fc114ee4202ab8db00474e4c7ed` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.321817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b39fc114ee4202ab8db00474e4c7ed)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.321817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.455843 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1265/2547] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:32 2022] PUT /identity/v3/projects/423a4b3ce644466fb21289009a186292/users/a193f92de13744a0a4b8cd4e9242f403/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:32.456697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.511497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b39fc114ee4202ab8db00474e4c7ed', 'request_id': 'req-eabece30-5dfb-4727-8b02-e06903bfbc59'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:32.518713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b39fc114ee4202ab8db00474e4c7ed', 'request_id': 'req-eabece30-5dfb-4727-8b02-e06903bfbc59'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:32.526019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe22cc-0aeb-4aca-8550-bb4d39ec7033 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.584099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd30d2-a8b8-4b81-a9a9-9540b5a9513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.584770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd30d2-a8b8-4b81-a9a9-9540b5a9513d admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.585074 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd30d2-a8b8-4b81-a9a9-9540b5a9513d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.585350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd30d2-a8b8-4b81-a9a9-9540b5a9513d admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.586076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd30d2-a8b8-4b81-a9a9-9540b5a9513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.587735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd30d2-a8b8-4b81-a9a9-9540b5a9513d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.598363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1266/2548] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:32.599693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca07b1-4873-4675-a624-27bdd17cb79b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.599942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca07b1-4873-4675-a624-27bdd17cb79b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.600222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca07b1-4873-4675-a624-27bdd17cb79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.621511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b39fc114ee4202ab8db00474e4c7ed', 'request_id': 'req-eabece30-5dfb-4727-8b02-e06903bfbc59'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:32.629883 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-0dca07b1-4873-4675-a624-27bdd17cb79b None None] MFA Rules not processed for user `a193f92de13744a0a4b8cd4e9242f403`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:32.631131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-0dca07b1-4873-4675-a624-27bdd17cb79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:32.633961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] The token cache is being invalidate because project 53b39fc114ee4202ab8db00474e4c7ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:32.634287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:32.634976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:32.636603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1283/2549] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:32 2022] DELETE /identity/v3/projects/53b39fc114ee4202ab8db00474e4c7ed => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:32.637353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.638851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:32.651928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabece30-5dfb-4727-8b02-e06903bfbc59 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.656877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfddabc-3f3d-47c8-9dc6-1aa0556dbf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.657761 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddabc-3f3d-47c8-9dc6-1aa0556dbf36 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.658097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddabc-3f3d-47c8-9dc6-1aa0556dbf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.658437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddabc-3f3d-47c8-9dc6-1aa0556dbf36 admin admin] PATH_INFO: `/v3/projects/7579559e83c248a991d90287b48f3ff1/users/e0054a0451e44ec89c556e71472ba11e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.674210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfddabc-3f3d-47c8-9dc6-1aa0556dbf36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7579559e83c248a991d90287b48f3ff1, user_id=e0054a0451e44ec89c556e71472ba11e, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.676945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfddabc-3f3d-47c8-9dc6-1aa0556dbf36 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.701632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1267/2550] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:32.703383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.703793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.704168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.724652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:32.725982 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:32.735089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1268/2551] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:32.735983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.742573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:32.757322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794190f-59f6-469a-bc78-0cf377a09355 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:32.762575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:32.763262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.763543 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.763816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.764509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:32.766445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:32.820914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1284/2552] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:32 2022] PUT /identity/v3/projects/7579559e83c248a991d90287b48f3ff1/users/e0054a0451e44ec89c556e71472ba11e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:32.822477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3420b1-ca2f-4c9f-b718-9f7095cdc74d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.822835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3420b1-ca2f-4c9f-b718-9f7095cdc74d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.823209 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3420b1-ca2f-4c9f-b718-9f7095cdc74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.847187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-8b3420b1-ca2f-4c9f-b718-9f7095cdc74d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:32.848528 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8b3420b1-ca2f-4c9f-b718-9f7095cdc74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:32.922342 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1285/2553] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:32.924396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe83a26-d6c0-4bc8-af28-49fe76f04586 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:32.924699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe83a26-d6c0-4bc8-af28-49fe76f04586 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:32.925019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe83a26-d6c0-4bc8-af28-49fe76f04586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:32.944909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1269/2554] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:32 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:32.946035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:32.962405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ebe83a26-d6c0-4bc8-af28-49fe76f04586 None None] MFA Rules not processed for user `e0054a0451e44ec89c556e71472ba11e`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:32.966122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ebe83a26-d6c0-4bc8-af28-49fe76f04586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:33.027189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c02bf6-1e51-488c-b158-ae6a4449c688 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.031698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.032368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.032653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.032930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.033662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:33.033973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.034679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1286/2555] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:33.035767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.038927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe83a26-d6c0-4bc8-af28-49fe76f04586 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.049728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1270/2556] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:04:33.049728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.050028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe83a26-d6c0-4bc8-af28-49fe76f04586 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.055969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.056641 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.056922 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.057197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.058248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.059115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08379db8-b6fb-464f-b8af-a49b3b3116a1 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.060238 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.063715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.064367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.064720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.065007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.065832 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.067888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.196943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1287/2557] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:33 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:33.197890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.207992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a7b8f-ed2c-49ee-a288-77b70eb1c8b3 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.212659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.213336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.213639 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.214488 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.214891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.217458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.234662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1271/2558] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:33 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:33.235501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.282997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:33.318899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d1261-97c6-4dee-94c0-d45c057852f2 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.325851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.326866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.327302 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.327746 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.328778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.330814 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.348141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1272/2559] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:33.348141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.356479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cf4c0-b872-46dc-933c-b38f3ae5771a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.361163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.361816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.362103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.362408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.363091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.365005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.388027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1288/2560] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:33 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:33.394473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.449279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66707e2e-146e-43d2-a7ae-9b32ca2e819a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.453926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.454864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.455150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.455427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] PATH_INFO: `/v3/projects/87112d0bbbee45cbbdd077817799dfab/users/bfad769a1eca41b79253e6e413de56d2/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.470994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] RBAC: Authorizing `identity:create_grant(project_id=87112d0bbbee45cbbdd077817799dfab, user_id=bfad769a1eca41b79253e6e413de56d2, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.473680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.571927 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1273/2561] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:33 2022] POST /identity/v3/users => generated 300 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:33.573730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.583684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900934c-65c7-41fd-aeb2-ee92948d5138 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.588443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.589303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.589667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.589950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.590669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.592639 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.606098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1289/2562] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:33 2022] PUT /identity/v3/projects/87112d0bbbee45cbbdd077817799dfab/users/bfad769a1eca41b79253e6e413de56d2/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:33.612509 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.671377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6538923-d6dd-4df6-89ca-7c48afd9bcda admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.676073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.676740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.677030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.677306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.678036 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.679679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.690351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1290/2563] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:33.691189 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.702016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1dcc-13e4-4c4d-b180-8abe6f58ea4e admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.706558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.707253 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.707538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.707817 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.708660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.710506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.720965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1291/2564] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:33.721855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.731823 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1ccb-9c8e-4770-b544-0d86ecf436ae admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.738416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.738416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.738416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.738416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] PATH_INFO: `/v3/projects/264fa8f3b5be40daaf849e5d544d02aa/users/9fb37eb0ddf143408dffd45ddbb2d059/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.754021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=264fa8f3b5be40daaf849e5d544d02aa, user_id=9fb37eb0ddf143408dffd45ddbb2d059, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.756896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.776371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1274/2565] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:33 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:33.784362 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.787099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf31dc-8977-471b-9a1f-73a38efe733a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.791785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.792667 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.792950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.793231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] PATH_INFO: `/v3/projects/87112d0bbbee45cbbdd077817799dfab/users/bfad769a1eca41b79253e6e413de56d2/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.795290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] RBAC: Authorizing `identity:create_grant(project_id=87112d0bbbee45cbbdd077817799dfab, user_id=bfad769a1eca41b79253e6e413de56d2, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.797991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.810479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1275/2566] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:33 2022] PUT /identity/v3/projects/87112d0bbbee45cbbdd077817799dfab/users/bfad769a1eca41b79253e6e413de56d2/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:33.811312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.882828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc4016-f3c8-44fc-b248-957233ee30be admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:33.887607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199034b5-1e3f-4dee-84df-70cbfaa49a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:33.888288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199034b5-1e3f-4dee-84df-70cbfaa49a58 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.888573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199034b5-1e3f-4dee-84df-70cbfaa49a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.888855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199034b5-1e3f-4dee-84df-70cbfaa49a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.889600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199034b5-1e3f-4dee-84df-70cbfaa49a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:33.891262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199034b5-1e3f-4dee-84df-70cbfaa49a58 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:33.904896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1276/2567] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:33.904896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82917d-40aa-465e-8b6d-dcdfe5c8b7fe None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:33.904896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82917d-40aa-465e-8b6d-dcdfe5c8b7fe None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:33.904896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82917d-40aa-465e-8b6d-dcdfe5c8b7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:33.910673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1292/2568] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:33 2022] PUT /identity/v3/projects/264fa8f3b5be40daaf849e5d544d02aa/users/9fb37eb0ddf143408dffd45ddbb2d059/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:33.910673 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:33.934116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-6c82917d-40aa-465e-8b6d-dcdfe5c8b7fe None None] MFA Rules not processed for user `bfad769a1eca41b79253e6e413de56d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:33.935273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-6c82917d-40aa-465e-8b6d-dcdfe5c8b7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:33.999170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bc3d0-e979-4272-9ced-39c8d687b442 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.006543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.007613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.007968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.008483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.009312 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.013202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.025893 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1293/2569] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:34.036620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.049692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1277/2570] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:34.050931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82917d-40aa-465e-8b6d-dcdfe5c8b7fe None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.056553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af4fa4-6fe7-454d-93de-48a4d4495187 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.063026 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82917d-40aa-465e-8b6d-dcdfe5c8b7fe None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.067672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.068076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.070421 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.070525 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.070606 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.070701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.070795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] PATH_INFO: `/v3/projects/6c9fc9a791494fc780ec1d201c52729b/users/41312733ecb44946a133a394138d897a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.070904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.070904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.071697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.083665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1294/2571] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:34.084765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.086502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:34.090167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9fc9a791494fc780ec1d201c52729b, user_id=41312733ecb44946a133a394138d897a, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.093192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.151187 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b9bbb-b420-4967-8740-4f34e32c50d4 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.155664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c6b6c-cb7f-4b33-9210-a9797e26b608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.156324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c6b6c-cb7f-4b33-9210-a9797e26b608 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.156608 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c6b6c-cb7f-4b33-9210-a9797e26b608 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.156884 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c6b6c-cb7f-4b33-9210-a9797e26b608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.166274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c6b6c-cb7f-4b33-9210-a9797e26b608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.169356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c6b6c-cb7f-4b33-9210-a9797e26b608 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.176804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1278/2572] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:34 2022] PUT /identity/v3/projects/6c9fc9a791494fc780ec1d201c52729b/users/41312733ecb44946a133a394138d897a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:34.179985 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1295/2573] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:34.187150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.188046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848b4da-1455-48da-bac2-df5b3ca2d4a8 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.188544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848b4da-1455-48da-bac2-df5b3ca2d4a8 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.189019 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848b4da-1455-48da-bac2-df5b3ca2d4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.223908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6848b4da-1455-48da-bac2-df5b3ca2d4a8 None None] MFA Rules not processed for user `41312733ecb44946a133a394138d897a`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:34.225211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6848b4da-1455-48da-bac2-df5b3ca2d4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:34.267216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301726b3-64d5-4f06-b019-260d8c9fbb5a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.272802 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdefdb1-188d-433d-9ce9-5d459479fda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.273865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdefdb1-188d-433d-9ce9-5d459479fda5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.274017 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdefdb1-188d-433d-9ce9-5d459479fda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.274340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdefdb1-188d-433d-9ce9-5d459479fda5 admin admin] PATH_INFO: `/v3/projects/264fa8f3b5be40daaf849e5d544d02aa/users/9fb37eb0ddf143408dffd45ddbb2d059/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.276697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdefdb1-188d-433d-9ce9-5d459479fda5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=264fa8f3b5be40daaf849e5d544d02aa, user_id=9fb37eb0ddf143408dffd45ddbb2d059, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.280145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdefdb1-188d-433d-9ce9-5d459479fda5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.295874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1279/2574] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:34 2022] PUT /identity/v3/projects/264fa8f3b5be40daaf849e5d544d02aa/users/9fb37eb0ddf143408dffd45ddbb2d059/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:34.309718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1296/2575] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:34.312599 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7cafc-3da0-47ea-8187-6e2142fc001e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.313195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7cafc-3da0-47ea-8187-6e2142fc001e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.313961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848b4da-1455-48da-bac2-df5b3ca2d4a8 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.314227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7cafc-3da0-47ea-8187-6e2142fc001e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.325731 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848b4da-1455-48da-bac2-df5b3ca2d4a8 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.351063 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-62c7cafc-3da0-47ea-8187-6e2142fc001e None None] MFA Rules not processed for user `9fb37eb0ddf143408dffd45ddbb2d059`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:34.358897 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-62c7cafc-3da0-47ea-8187-6e2142fc001e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:34.411399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.412096 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.412414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.412699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.413395 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.415353 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.438775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1280/2576] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:34.450551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7cafc-3da0-47ea-8187-6e2142fc001e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.506777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7cafc-3da0-47ea-8187-6e2142fc001e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.510481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.511132 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.511416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.511693 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.521079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.522589 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.533402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1281/2577] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:34.550449 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1297/2578] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:34 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:34.551208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.565153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb6c67-f543-4b66-9ad1-c12d83f2400f service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.565304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.567169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:34.589042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:34.642129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.643001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.643292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.643573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.644275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.646485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.663869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7fc0f-82b6-4378-9c9f-0b1853ac5035 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.667762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.668659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.668954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.669239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.730092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.731781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.741549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1298/2579] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:34.742234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.754489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649af2c-f065-493e-9262-6fc415acc259 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.759350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.760239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.761264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.761551 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] PATH_INFO: `/v3/users/e0054a0451e44ec89c556e71472ba11e` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.762849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0054a0451e44ec89c556e71472ba11e)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.764807 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.798619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1282/2580] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:34 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:34.802492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.812740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d342d-52a1-4a4c-858c-233a1d5a23db admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.817600 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.818337 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.818620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.818974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.819764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.821472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:34.832983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1283/2581] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:34.837076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:34.847665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f199b3-f914-449e-9393-c2cb24465d5d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:34.852635 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:34.857602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:34.857602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:34.857602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] PATH_INFO: `/v3/projects/c4c611de21884f8b8e6514069c9e5001/users/31a36f13d29a44148b8c95b0981119c8/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:34.884280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c611de21884f8b8e6514069c9e5001, user_id=31a36f13d29a44148b8c95b0981119c8, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:34.887156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:35.058155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1284/2582] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:34 2022] PUT /identity/v3/projects/c4c611de21884f8b8e6514069c9e5001/users/31a36f13d29a44148b8c95b0981119c8/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:35.062425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:35.165765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0054a0451e44ec89c556e71472ba11e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:35.167380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0054a0451e44ec89c556e71472ba11e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:35.178659 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0054a0451e44ec89c556e71472ba11e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:35.208402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a54001-b713-43db-bdfb-939f9553f35e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:35.214767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:35.215700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.215700 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.215999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.216734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:35.218434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:35.233134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1285/2583] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:35.234322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:35.236256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:35.249876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50374a-2b20-4011-a625-3cbecb930d7c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:35.254910 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455e0a-0a24-4ad2-adff-803ac6031a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:35.255822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455e0a-0a24-4ad2-adff-803ac6031a9a admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.256102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455e0a-0a24-4ad2-adff-803ac6031a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.256375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455e0a-0a24-4ad2-adff-803ac6031a9a admin admin] PATH_INFO: `/v3/projects/c4c611de21884f8b8e6514069c9e5001/users/31a36f13d29a44148b8c95b0981119c8/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.258563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca455e0a-0a24-4ad2-adff-803ac6031a9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c611de21884f8b8e6514069c9e5001, user_id=31a36f13d29a44148b8c95b0981119c8, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:35.262066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca455e0a-0a24-4ad2-adff-803ac6031a9a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:35.272686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1286/2584] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:35 2022] PUT /identity/v3/projects/c4c611de21884f8b8e6514069c9e5001/users/31a36f13d29a44148b8c95b0981119c8/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:35.287579 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.288139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.288425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.322426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] MFA Rules not processed for user `31a36f13d29a44148b8c95b0981119c8`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:35.324375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:35.340949 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0054a0451e44ec89c556e71472ba11e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:35.361838 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1a4c1584-94c1-4531-b57d-1bb4f3955889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0054a0451e44ec89c556e71472ba11e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:35.369118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1299/2585] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:34 2022] DELETE /identity/v3/users/e0054a0451e44ec89c556e71472ba11e => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:35.373990 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.374316 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.374597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.396475 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:35.397754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:35.426089 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1287/2586] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:35.448743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:35.451788 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:35.522202 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1300/2587] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:35.531834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:35.533944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:35.558257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ffe92-2f7d-4348-af23-78661663c308 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:35.621324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d51da-c039-4d30-b806-c86fe6b836d7 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:35.625172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:35.625962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.626382 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.626733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.628706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:35.629549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.629792 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.630396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.639389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:35.640991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:35.651796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1301/2588] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:35.660661 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:35.663691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:35.696419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e809a5-0e0d-4732-9fcd-0f6e07706114 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:35.702156 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:35.703666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:35.722826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055873d5-1e50-4b63-98ab-8a1586e8dd9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:35.724538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055873d5-1e50-4b63-98ab-8a1586e8dd9b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.725201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1288/2589] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:35.727538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055873d5-1e50-4b63-98ab-8a1586e8dd9b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.736063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055873d5-1e50-4b63-98ab-8a1586e8dd9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.740693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-055873d5-1e50-4b63-98ab-8a1586e8dd9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:35.745653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:35.766332 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb73e4-0bef-4097-8ea4-aa03d0ac7652 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:35.782284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:35.783568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:35.785777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:35.785777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] PATH_INFO: `/v3/users/41312733ecb44946a133a394138d897a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:35.786570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055873d5-1e50-4b63-98ab-8a1586e8dd9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:35.787304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41312733ecb44946a133a394138d897a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:35.788039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055873d5-1e50-4b63-98ab-8a1586e8dd9b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:35.790406 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:35.811150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1302/2590] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:36.142956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41312733ecb44946a133a394138d897a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.143572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41312733ecb44946a133a394138d897a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.144130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41312733ecb44946a133a394138d897a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.161903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41312733ecb44946a133a394138d897a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.243647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41312733ecb44946a133a394138d897a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.259711 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1289/2591] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:35 2022] DELETE /identity/v3/users/41312733ecb44946a133a394138d897a => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:36.264968 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:36.265420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:36.265831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:36.301099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:36.303103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:36.310836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:36.315119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:36.399991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36243e-cbec-4983-95dc-323b69d65d1f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:36.403973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:36.412199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:36.413311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:36.413709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:36.414197 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] PATH_INFO: `/v3/projects/7579559e83c248a991d90287b48f3ff1` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:36.415871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7579559e83c248a991d90287b48f3ff1)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:36.419869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:36.506854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1303/2592] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:36.520367 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:36.522988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:36.617734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7579559e83c248a991d90287b48f3ff1', 'request_id': 'req-5305e4eb-163c-4bf5-885f-2478cfa55bd0'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.644451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee4c06-201f-4b83-8fe1-d193bbdfda67 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:36.715274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:36.716201 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:36.716412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:36.716540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:36.723769 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7579559e83c248a991d90287b48f3ff1', 'request_id': 'req-5305e4eb-163c-4bf5-885f-2478cfa55bd0'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.732673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7579559e83c248a991d90287b48f3ff1', 'request_id': 'req-5305e4eb-163c-4bf5-885f-2478cfa55bd0'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.748376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] The token cache is being invalidate because project 7579559e83c248a991d90287b48f3ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:36.748647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.749459 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:36.751510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1290/2593] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:36 2022] DELETE /identity/v3/projects/7579559e83c248a991d90287b48f3ff1 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:36.756392 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:36.758458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:36.815166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:36.817636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:36.832102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1304/2594] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:36.893445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305e4eb-163c-4bf5-885f-2478cfa55bd0 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:36.898938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:36.899836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:36.900116 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:36.900386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] PATH_INFO: `/v3/users/9fb37eb0ddf143408dffd45ddbb2d059` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:36.902617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb37eb0ddf143408dffd45ddbb2d059)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:36.904887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:37.201840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb37eb0ddf143408dffd45ddbb2d059'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.202921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb37eb0ddf143408dffd45ddbb2d059'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.204506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb37eb0ddf143408dffd45ddbb2d059'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.228544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb37eb0ddf143408dffd45ddbb2d059'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.329772 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:37.330158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-17f9f501-ac2a-4d78-be21-f31c1da68acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb37eb0ddf143408dffd45ddbb2d059'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.343605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1291/2595] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:36 2022] DELETE /identity/v3/users/9fb37eb0ddf143408dffd45ddbb2d059 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:37.451218 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:37.453008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:37.468378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5b535-5069-4013-ab6d-0e0aebc473ae service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:37.550967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:37.551862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:37.552133 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:37.552410 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] PATH_INFO: `/v3/projects/6c9fc9a791494fc780ec1d201c52729b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:37.554134 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9fc9a791494fc780ec1d201c52729b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:37.556324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:37.661981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:37.661981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:37.661981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:37.677633 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] MFA Rules not processed for user `71f3bb1357f14481bd1850c1153bc253`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:37.679866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:37.746341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9fc9a791494fc780ec1d201c52729b', 'request_id': 'req-b494f708-ff7c-47b2-bf70-98d26d7850f0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.753974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1292/2596] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:37.759344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9fc9a791494fc780ec1d201c52729b', 'request_id': 'req-b494f708-ff7c-47b2-bf70-98d26d7850f0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.951857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9fc9a791494fc780ec1d201c52729b', 'request_id': 'req-b494f708-ff7c-47b2-bf70-98d26d7850f0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.969993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] The token cache is being invalidate because project 6c9fc9a791494fc780ec1d201c52729b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:37.970319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.971363 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:37.975042 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1305/2597] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:37 2022] DELETE /identity/v3/projects/6c9fc9a791494fc780ec1d201c52729b => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:37.983851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:37.985890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:38.014934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2b1fc-1eee-416d-aa65-dbc9b426f8a6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:38.085516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:38.086642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:38.087051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:38.091760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:38.093498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:38.095527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] PATH_INFO: `/v3/users/31a36f13d29a44148b8c95b0981119c8` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:38.098060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a36f13d29a44148b8c95b0981119c8)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:38.101850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:38.108994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494f708-ff7c-47b2-bf70-98d26d7850f0 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:38.115408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:38.116664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:38.117244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:38.117804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] PATH_INFO: `/v3/projects/264fa8f3b5be40daaf849e5d544d02aa` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:38.119521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=264fa8f3b5be40daaf849e5d544d02aa)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:38.122111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:38.412424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a36f13d29a44148b8c95b0981119c8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.414306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a36f13d29a44148b8c95b0981119c8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.415188 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a36f13d29a44148b8c95b0981119c8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.416455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264fa8f3b5be40daaf849e5d544d02aa', 'request_id': 'req-a43ab659-c28a-4a20-a86d-dce60a51af44'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.426704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264fa8f3b5be40daaf849e5d544d02aa', 'request_id': 'req-a43ab659-c28a-4a20-a86d-dce60a51af44'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.433104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a36f13d29a44148b8c95b0981119c8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.546849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264fa8f3b5be40daaf849e5d544d02aa', 'request_id': 'req-a43ab659-c28a-4a20-a86d-dce60a51af44'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.561095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] The token cache is being invalidate because project 264fa8f3b5be40daaf849e5d544d02aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:38.561689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.562594 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-a43ab659-c28a-4a20-a86d-dce60a51af44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.564903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1306/2598] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:38 2022] DELETE /identity/v3/projects/264fa8f3b5be40daaf849e5d544d02aa => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:38.569901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d7c56-f4bd-45a5-a6e6-37b7c00cd1f7 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:38.570458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d7c56-f4bd-45a5-a6e6-37b7c00cd1f7 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:38.570945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d7c56-f4bd-45a5-a6e6-37b7c00cd1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:38.598206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-ed6d7c56-f4bd-45a5-a6e6-37b7c00cd1f7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:38.605033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ed6d7c56-f4bd-45a5-a6e6-37b7c00cd1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:38.678416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1307/2599] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:38.684136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2a215-3a75-44b7-9b2a-89e8af07d532 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:38.684656 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2a215-3a75-44b7-9b2a-89e8af07d532 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:38.685192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2a215-3a75-44b7-9b2a-89e8af07d532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:38.706163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a36f13d29a44148b8c95b0981119c8'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:38.706626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-02b2a215-3a75-44b7-9b2a-89e8af07d532 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:38.707879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-02b2a215-3a75-44b7-9b2a-89e8af07d532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:38.717417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1308/2600] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:38.721664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2a215-3a75-44b7-9b2a-89e8af07d532 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:38.723835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1293/2601] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:37 2022] DELETE /identity/v3/users/31a36f13d29a44148b8c95b0981119c8 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:38.731321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2a215-3a75-44b7-9b2a-89e8af07d532 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:38.802118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:38.802835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:38.803095 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:38.803296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:38.804056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:38.804374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:38.806240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:38.819843 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1309/2602] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:04:38.824312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:38.826503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:38.841139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d723e7-a1be-47bf-bbc2-4a8ebb3581b3 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:38.846736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:38.847409 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:38.847697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:38.847977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:38.848684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:38.850665 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:38.993503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1294/2603] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:38 2022] POST /identity/v3/projects => generated 391 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:38.995049 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:39.009613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02119de6-99b1-4630-847c-d07ee9afdba4 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:39.176520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:39.177549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.178050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.178544 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.179393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:39.181494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:39.352085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1310/2604] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:38 2022] POST /identity/v3/users => generated 296 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:39.353340 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:39.365977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39edb22-70cb-4612-9036-1986a7097806 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:39.371946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:39.372674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.372990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.373529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.374305 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:39.375990 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:39.387461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1295/2605] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:39.391693 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:39.402903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:39.405110 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:39.408168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91429-1b7f-4f2b-a28d-e38273ce116f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:39.424474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:39.433621 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:39.434398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.434737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.434946 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.436644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:39.437478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7314ef0-3b31-4ee2-af16-7c49a7880773 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:39.439630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:39.451188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1311/2606] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:39.451996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:39.452993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.453458 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.453875 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] PATH_INFO: `/v3/projects/c4c611de21884f8b8e6514069c9e5001` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.455269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:39.455688 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4c611de21884f8b8e6514069c9e5001)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:39.457620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:39.466254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90f34d-c7e4-415d-8d7f-bbd4fce573a6 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:39.471419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dd77e-e3fd-4d96-af47-295e40be42e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:39.472411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dd77e-e3fd-4d96-af47-295e40be42e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.472763 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dd77e-e3fd-4d96-af47-295e40be42e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.473138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dd77e-e3fd-4d96-af47-295e40be42e2 admin admin] PATH_INFO: `/v3/projects/dc5d85e162a84e4aa383f6c3bcb985d5/users/750d5230610a4795b692a97556f16184/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.492117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83dd77e-e3fd-4d96-af47-295e40be42e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5d85e162a84e4aa383f6c3bcb985d5, user_id=750d5230610a4795b692a97556f16184, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:39.495917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83dd77e-e3fd-4d96-af47-295e40be42e2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:39.620279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1312/2607] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:39 2022] PUT /identity/v3/projects/dc5d85e162a84e4aa383f6c3bcb985d5/users/750d5230610a4795b692a97556f16184/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:39.630803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857efa4d-1efd-4e2b-b36c-2459ab6c33f9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.631211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857efa4d-1efd-4e2b-b36c-2459ab6c33f9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.631573 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857efa4d-1efd-4e2b-b36c-2459ab6c33f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.632850 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4c611de21884f8b8e6514069c9e5001', 'request_id': 'req-7fcfd5fb-7415-4c9c-b211-449c25331a8f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:39.660249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-857efa4d-1efd-4e2b-b36c-2459ab6c33f9 None None] MFA Rules not processed for user `750d5230610a4795b692a97556f16184`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:39.661473 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-857efa4d-1efd-4e2b-b36c-2459ab6c33f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:39.728198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1313/2608] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:39.733587 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857efa4d-1efd-4e2b-b36c-2459ab6c33f9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:39.797572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857efa4d-1efd-4e2b-b36c-2459ab6c33f9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:39.799279 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:39.799952 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.800232 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.800511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.801244 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:39.803252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:39.862919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4c611de21884f8b8e6514069c9e5001', 'request_id': 'req-7fcfd5fb-7415-4c9c-b211-449c25331a8f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:39.872828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4c611de21884f8b8e6514069c9e5001', 'request_id': 'req-7fcfd5fb-7415-4c9c-b211-449c25331a8f'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:39.894780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] The token cache is being invalidate because project c4c611de21884f8b8e6514069c9e5001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:39.895080 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:39.898542 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-7fcfd5fb-7415-4c9c-b211-449c25331a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:39.905976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1296/2609] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:39 2022] DELETE /identity/v3/projects/c4c611de21884f8b8e6514069c9e5001 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:39.908166 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76bea1-e928-43bf-b89c-7402eee04b42 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:39.908488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76bea1-e928-43bf-b89c-7402eee04b42 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:39.908782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76bea1-e928-43bf-b89c-7402eee04b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:39.973529 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-cc76bea1-e928-43bf-b89c-7402eee04b42 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:39.976168 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cc76bea1-e928-43bf-b89c-7402eee04b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:39.990771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1314/2610] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:39 2022] POST /identity/v3/projects => generated 393 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:39.997738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.016626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:40.073508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1297/2611] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:40.075366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940f89-c86c-4271-b84c-5b78a4abdfab None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.075886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940f89-c86c-4271-b84c-5b78a4abdfab None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.076455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5940f89-c86c-4271-b84c-5b78a4abdfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.106831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-c5940f89-c86c-4271-b84c-5b78a4abdfab None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:40.106831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-c5940f89-c86c-4271-b84c-5b78a4abdfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:40.119668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1298/2612] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:40.120981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5940f89-c86c-4271-b84c-5b78a4abdfab None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.134439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5940f89-c86c-4271-b84c-5b78a4abdfab None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.149233 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.150057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.152129 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.152860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.154939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:40.158102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.160580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.184160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1299/2613] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:04:40.190376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.192358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fcf55-bcba-4dbf-a4fc-1c05f8448139 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.205033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.205732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.206015 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.206321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.206815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43cc3e-f68f-4750-80a5-d4adbdff4556 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.207018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.210977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.215144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.216336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.226873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.227158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.228158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.231696 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.342029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1300/2614] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:40 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:40.343953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1315/2615] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:39 2022] POST /identity/v3/users => generated 303 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:40.349355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.349537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.363399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b831a-3c0f-48a8-8cc1-58e766ff540d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.439083 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e06fc8-fa4d-46f6-85d2-9a35182947fa admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.444256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.444955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.445308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.445435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.445666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.446767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.449534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.454846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.455145 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.462643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.462643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.463175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1316/2616] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:40.465578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.466988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.479255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a43a5b-3b86-4c1c-970f-3c3f6fc45a24 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.501593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.503235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.505384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.505798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] PATH_INFO: `/v3/projects/09c507a0f3dc41a5a0caabdf3fc4057a/users/74b45956d6484784b32f9467615f804a/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.558574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c507a0f3dc41a5a0caabdf3fc4057a, user_id=74b45956d6484784b32f9467615f804a, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.576782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.603597 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1301/2617] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:40 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:40.610082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.628199 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8496e5-3007-4ef1-be0b-c21142f6b7ae admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.633712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.636909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.636909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.636909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.636909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.638745 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.652426 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1302/2618] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:40.656610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.673334 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114a1e1-d984-4085-89e5-1e8510cd34ce admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.678792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1317/2619] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:40 2022] PUT /identity/v3/projects/09c507a0f3dc41a5a0caabdf3fc4057a/users/74b45956d6484784b32f9467615f804a/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:40.684860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.684860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:40.803871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.805658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.806774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.807025 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.808339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.810211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.822088 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1303/2620] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:40.823109 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.825369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a30ba-15db-414c-b74a-b479ba374007 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.839609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:40.839831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.839831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.839831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.839831 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.840207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.841076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.855849 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1318/2621] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:40.863120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:40.875269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36f639-129a-4005-815b-e8b6f5a0012a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.880411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b10a8-90fb-4bb7-8d68-44a4ca19d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.881275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b10a8-90fb-4bb7-8d68-44a4ca19d437 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.881561 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b10a8-90fb-4bb7-8d68-44a4ca19d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.881837 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b10a8-90fb-4bb7-8d68-44a4ca19d437 admin admin] PATH_INFO: `/v3/projects/5ad6ec15e5214485944d94f74c7c0704/users/7291778f62e9451c93f8760d6d21b12f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.900644 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b10a8-90fb-4bb7-8d68-44a4ca19d437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad6ec15e5214485944d94f74c7c0704, user_id=7291778f62e9451c93f8760d6d21b12f, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.903944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b10a8-90fb-4bb7-8d68-44a4ca19d437 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:40.910457 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17d44e-5a54-4b14-8843-50d0ffae7d32 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:40.914873 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:40.925595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:40.925595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:40.925595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:40.925595 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:40.988697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:40.990308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.000857 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1304/2622] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:41.001761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.012614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f17a80-2d59-4136-b0e4-267dd5a857b3 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.023124 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc2a08-6458-4b36-9f59-08cd99b03da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.024648 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc2a08-6458-4b36-9f59-08cd99b03da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.025121 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc2a08-6458-4b36-9f59-08cd99b03da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.025613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc2a08-6458-4b36-9f59-08cd99b03da2 admin admin] PATH_INFO: `/v3/projects/09c507a0f3dc41a5a0caabdf3fc4057a/users/74b45956d6484784b32f9467615f804a/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.037523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc2a08-6458-4b36-9f59-08cd99b03da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c507a0f3dc41a5a0caabdf3fc4057a, user_id=74b45956d6484784b32f9467615f804a, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:41.042712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc2a08-6458-4b36-9f59-08cd99b03da2 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.043972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1319/2623] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:40 2022] PUT /identity/v3/projects/5ad6ec15e5214485944d94f74c7c0704/users/7291778f62e9451c93f8760d6d21b12f/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:41.063306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.063858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.064396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.070001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1305/2624] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:41 2022] PUT /identity/v3/projects/09c507a0f3dc41a5a0caabdf3fc4057a/users/74b45956d6484784b32f9467615f804a/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:41.094000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9108f4a-6736-444f-9458-37e7e293c851 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.094760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9108f4a-6736-444f-9458-37e7e293c851 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.095222 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9108f4a-6736-444f-9458-37e7e293c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.123435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] MFA Rules not processed for user `7291778f62e9451c93f8760d6d21b12f`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:41.123435 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:41.158869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-f9108f4a-6736-444f-9458-37e7e293c851 None None] MFA Rules not processed for user `74b45956d6484784b32f9467615f804a`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:41.166513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f9108f4a-6736-444f-9458-37e7e293c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:41.303774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1306/2625] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:41.310892 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1320/2626] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:41.315385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9108f4a-6736-444f-9458-37e7e293c851 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.318372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.322655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:41.443757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce6fab-542c-4515-8c39-0d91436b6399 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.444323 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9108f4a-6736-444f-9458-37e7e293c851 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.447709 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.448373 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.448651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.448972 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.453912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.454955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.455651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.456005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.456967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:41.459507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.467098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:41.467287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:41.467401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.477534 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1321/2627] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:41.534519 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.536275 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:41.555942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dd98f-54eb-4139-9807-e4f4efdd5e82 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.560370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.561053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.561417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.561689 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.563540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:41.634288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:41.636443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.645792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1322/2628] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:41.702853 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1307/2629] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:41 2022] POST /identity/v3/projects => generated 377 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:41.703409 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.715480 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22dfe1-76ac-4a75-bf4c-b43c61a8fb83 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.788880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.789566 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.789851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.790128 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.791053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:41.793872 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.886358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1323/2630] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:41 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:41.890743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.902821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30038789-0bdc-4b6d-9fe6-3bba2da5074f admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.907770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.908438 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.908778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.909056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.909786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:41.911455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.926498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1308/2631] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:41.927865 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.938839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d14b0f-2ecd-4c44-bc52-a1e0bbeafb29 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.943554 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36769316-fee5-47d6-93c8-049774fe1d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.944322 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36769316-fee5-47d6-93c8-049774fe1d65 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.944678 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36769316-fee5-47d6-93c8-049774fe1d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.945032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36769316-fee5-47d6-93c8-049774fe1d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:41.945830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36769316-fee5-47d6-93c8-049774fe1d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:41.947654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36769316-fee5-47d6-93c8-049774fe1d65 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:41.958648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1324/2632] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:41.962778 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:41.974217 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c28d0-fb4f-4b22-b912-064047a3bcf3 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:41.988149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:41.989777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:41.989777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:41.989777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] PATH_INFO: `/v3/projects/1e2d9a2c7328491c88cb60dc541f5fdc/users/2ccfd9270fc041bfbceb92a32846e789/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:42.007954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2d9a2c7328491c88cb60dc541f5fdc, user_id=2ccfd9270fc041bfbceb92a32846e789, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:42.013082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:42.135727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1309/2633] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:41 2022] PUT /identity/v3/projects/1e2d9a2c7328491c88cb60dc541f5fdc/users/2ccfd9270fc041bfbceb92a32846e789/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:42.147173 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df96e-b700-4efe-9e9b-1e3dcb2f5655 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:42.147711 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df96e-b700-4efe-9e9b-1e3dcb2f5655 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:42.148247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df96e-b700-4efe-9e9b-1e3dcb2f5655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:42.186701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-850df96e-b700-4efe-9e9b-1e3dcb2f5655 None None] MFA Rules not processed for user `2ccfd9270fc041bfbceb92a32846e789`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:42.187864 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-850df96e-b700-4efe-9e9b-1e3dcb2f5655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:42.270955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1325/2634] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:42.283091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:42.353627 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c7171-8ee0-4f1d-ae63-32059af11262 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:42.359267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:42.359934 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:42.360219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:42.360494 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:42.361195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:42.363349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:42.533504 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1310/2635] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:42 2022] POST /identity/v3/projects => generated 377 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:42.537138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850df96e-b700-4efe-9e9b-1e3dcb2f5655 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:42.547717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850df96e-b700-4efe-9e9b-1e3dcb2f5655 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:42.639039 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:42.640487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:42.641050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:42.641393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:42.642151 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:42.645259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:42.794844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:42.804482 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1326/2636] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:42 2022] POST /identity/v3/users => generated 295 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:42.805574 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:42.819044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d872a-393e-441f-955c-b2b54c54c7a7 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:42.824808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:42.825585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:42.830702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:42.830702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:42.830702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:42.830702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:42.844993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1311/2637] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:42.846318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:42.857583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e36c49-456c-4489-ac3d-0b7601195487 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:42.864073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:42.865381 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:42.865662 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:42.866024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] PATH_INFO: `/v3/projects/e9f6b6f98e5d4bc5884d70bd48aaff45/users/e2b02caafb5d401280fc667143dafe65/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:42.885479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f6b6f98e5d4bc5884d70bd48aaff45, user_id=e2b02caafb5d401280fc667143dafe65, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:42.888310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:42.986996 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1327/2638] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:42 2022] PUT /identity/v3/projects/e9f6b6f98e5d4bc5884d70bd48aaff45/users/e2b02caafb5d401280fc667143dafe65/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:42.991091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:43.002320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f46b05-8ad2-4e3e-8362-392611d6d901 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:43.060828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e194e6-9a1b-4a5c-8036-b30f45456a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:43.062284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e194e6-9a1b-4a5c-8036-b30f45456a62 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:43.062284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e194e6-9a1b-4a5c-8036-b30f45456a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:43.062284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e194e6-9a1b-4a5c-8036-b30f45456a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:43.062861 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e194e6-9a1b-4a5c-8036-b30f45456a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:43.064485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e194e6-9a1b-4a5c-8036-b30f45456a62 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:43.074785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1312/2639] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:43.078889 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:43.089564 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f649143-4d60-43ac-a4df-ebef4d057a08 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:43.094697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:43.095585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:43.095874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:43.096155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] PATH_INFO: `/v3/projects/e9f6b6f98e5d4bc5884d70bd48aaff45/users/e2b02caafb5d401280fc667143dafe65/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:43.098154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f6b6f98e5d4bc5884d70bd48aaff45, user_id=e2b02caafb5d401280fc667143dafe65, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:43.100830 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:43.113694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1328/2640] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:43 2022] PUT /identity/v3/projects/e9f6b6f98e5d4bc5884d70bd48aaff45/users/e2b02caafb5d401280fc667143dafe65/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:43.124570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488a191-bf69-4d42-900e-49b31d3fd3c7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:43.124867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488a191-bf69-4d42-900e-49b31d3fd3c7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:43.125297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488a191-bf69-4d42-900e-49b31d3fd3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:43.155270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3488a191-bf69-4d42-900e-49b31d3fd3c7 None None] MFA Rules not processed for user `e2b02caafb5d401280fc667143dafe65`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:43.156565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3488a191-bf69-4d42-900e-49b31d3fd3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:43.227388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1313/2641] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:43.239237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:43.295145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778d08d-b2ac-421c-beb8-3d933dbd611a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:43.299047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:43.299704 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:43.299995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:43.300274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:43.311903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:43.313380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:43.330380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1329/2642] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:43.447766 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488a191-bf69-4d42-900e-49b31d3fd3c7 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:43.459908 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488a191-bf69-4d42-900e-49b31d3fd3c7 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:43.471749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:43.473584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:43.474604 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:43.475455 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:43.550091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:43.553416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:43.561239 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1314/2643] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:43.763767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:44.945885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:45.901483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:47.812299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:48.665403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:48.668625 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:48.745928 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaa91c-4799-4df1-bead-7bb2411bf857 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:48.751405 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:48.752318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:48.752651 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:48.752932 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] PATH_INFO: `/v3/users/26608323a21f4cb7832b93bdfd971663` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:48.754165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=26608323a21f4cb7832b93bdfd971663)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:48.756846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:49.313980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26608323a21f4cb7832b93bdfd971663'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:49.314717 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26608323a21f4cb7832b93bdfd971663'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:49.323945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26608323a21f4cb7832b93bdfd971663'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:49.505905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26608323a21f4cb7832b93bdfd971663'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:49.520888 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26608323a21f4cb7832b93bdfd971663'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:49.527734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1330/2644] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:48 2022] DELETE /identity/v3/users/26608323a21f4cb7832b93bdfd971663 => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:50.028397 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:50.040652 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206197e9-3f87-4243-9c67-7a3e5f55c542 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:50.098950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:50.099820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:50.100105 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:50.100384 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] PATH_INFO: `/v3/projects/06d1766d477e4d3993d1474d6e1a30e6` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:50.101554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d1766d477e4d3993d1474d6e1a30e6)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:50.104057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:50.123441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:50.310100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d1766d477e4d3993d1474d6e1a30e6', 'request_id': 'req-4aed1b49-fd60-4574-988c-f0d57001b0da'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:50.465782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d1766d477e4d3993d1474d6e1a30e6', 'request_id': 'req-4aed1b49-fd60-4574-988c-f0d57001b0da'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:50.475134 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d1766d477e4d3993d1474d6e1a30e6', 'request_id': 'req-4aed1b49-fd60-4574-988c-f0d57001b0da'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:50.495131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] The token cache is being invalidate because project 06d1766d477e4d3993d1474d6e1a30e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:50.495613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:50.496614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:50.499307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1315/2645] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:50 2022] DELETE /identity/v3/projects/06d1766d477e4d3993d1474d6e1a30e6 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:50.504679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:50.506198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:50.519791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:50.521230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:50.527443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f27b-c573-4f1c-ab34-419c79fc6adc admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:50.604250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:50.607243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:50.607660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:50.607944 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] PATH_INFO: `/v3/users/dfd0e1ae917d49c8839c36c50d41217d` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:50.609293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd0e1ae917d49c8839c36c50d41217d)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:50.610400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed1b49-fd60-4574-988c-f0d57001b0da admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:50.611802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:50.617312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:50.618655 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:50.619452 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:50.619929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] PATH_INFO: `/v3/users/750d5230610a4795b692a97556f16184` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:50.621300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750d5230610a4795b692a97556f16184)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:50.623794 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:51.271674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd0e1ae917d49c8839c36c50d41217d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.271674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd0e1ae917d49c8839c36c50d41217d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.279590 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd0e1ae917d49c8839c36c50d41217d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.283726 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750d5230610a4795b692a97556f16184'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.284803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750d5230610a4795b692a97556f16184'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.285488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750d5230610a4795b692a97556f16184'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.299975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750d5230610a4795b692a97556f16184'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.442494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd0e1ae917d49c8839c36c50d41217d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.460267 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd0e1ae917d49c8839c36c50d41217d'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.466030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1331/2646] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:50 2022] DELETE /identity/v3/users/dfd0e1ae917d49c8839c36c50d41217d => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:51.467601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:51.470417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:51.572520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf42f9a-0ca3-4262-a27b-faec06eb454f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:51.579172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:51.580299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:51.580737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:51.581247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] PATH_INFO: `/v3/users/7291778f62e9451c93f8760d6d21b12f` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:51.582891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7291778f62e9451c93f8760d6d21b12f)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:51.585126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:51.647672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-34cac2f1-5641-4aba-a405-6095f8e18968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750d5230610a4795b692a97556f16184'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:51.662965 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1316/2647] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:50 2022] DELETE /identity/v3/users/750d5230610a4795b692a97556f16184 => generated 0 bytes in 1143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:51.668029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:51.668350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:51.668632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:51.694242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:51.696038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:51.765505 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1317/2648] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:51.775851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:51.777084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:51.846016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea24cab-488b-4f35-bca9-f491c36b4997 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:51.849500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:51.850161 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:51.850475 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:51.850749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:51.862911 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:51.864442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:51.878374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1318/2649] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:51.932859 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:51.935281 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:51.962642 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387838d-51c3-4f59-9538-bc7a4c7cacae service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:51.967644 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:51.969815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:51.970215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:51.970429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] PATH_INFO: `/v3/projects/be09bb633e2549568d645fbfcf6aafa5` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:51.972055 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=be09bb633e2549568d645fbfcf6aafa5)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:51.974544 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:52.053247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7291778f62e9451c93f8760d6d21b12f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.054089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7291778f62e9451c93f8760d6d21b12f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.061445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7291778f62e9451c93f8760d6d21b12f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.235977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7291778f62e9451c93f8760d6d21b12f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.255470 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-85b68434-0b3c-4557-ad0a-6cffe65f7239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7291778f62e9451c93f8760d6d21b12f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.265044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be09bb633e2549568d645fbfcf6aafa5', 'request_id': 'req-01b502b6-cea1-401c-9528-b43de3ce4fac'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.267399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1332/2650] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:51 2022] DELETE /identity/v3/users/7291778f62e9451c93f8760d6d21b12f => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:52.272038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:52.272336 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:52.272619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:52.307570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:52.309579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:52.375921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1333/2651] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:52.385982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:52.387441 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:52.391284 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be09bb633e2549568d645fbfcf6aafa5', 'request_id': 'req-01b502b6-cea1-401c-9528-b43de3ce4fac'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.398424 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be09bb633e2549568d645fbfcf6aafa5', 'request_id': 'req-01b502b6-cea1-401c-9528-b43de3ce4fac'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.410378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] The token cache is being invalidate because project be09bb633e2549568d645fbfcf6aafa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:52.410666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.411994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.413702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1319/2652] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:51 2022] DELETE /identity/v3/projects/be09bb633e2549568d645fbfcf6aafa5 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:52.458257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b997ad-ea87-4bf7-8a47-312130150126 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:52.458936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:52.460492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:52.461987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:52.463211 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:52.464158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:52.464462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:52.466507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:52.481850 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:52.483428 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:52.499939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1334/2653] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:52.538297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b502b6-cea1-401c-9528-b43de3ce4fac admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:52.543840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:52.544743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:52.545023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:52.545295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] PATH_INFO: `/v3/users/a193f92de13744a0a4b8cd4e9242f403` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:52.546568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a193f92de13744a0a4b8cd4e9242f403)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:52.553683 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:52.571649 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:52.578807 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:52.594723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3648d1-e258-4e57-be7e-e8f353e834f3 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:52.609526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:52.616868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:52.617165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:52.617446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] PATH_INFO: `/v3/projects/dc5d85e162a84e4aa383f6c3bcb985d5` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:52.623714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5d85e162a84e4aa383f6c3bcb985d5)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:52.626154 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:52.921226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5d85e162a84e4aa383f6c3bcb985d5', 'request_id': 'req-f0196be6-8d59-4487-815f-95f23ed59dcf'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.921996 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a193f92de13744a0a4b8cd4e9242f403'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.923047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a193f92de13744a0a4b8cd4e9242f403'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.923730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a193f92de13744a0a4b8cd4e9242f403'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.927767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5d85e162a84e4aa383f6c3bcb985d5', 'request_id': 'req-f0196be6-8d59-4487-815f-95f23ed59dcf'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:52.937740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a193f92de13744a0a4b8cd4e9242f403'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.012756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5d85e162a84e4aa383f6c3bcb985d5', 'request_id': 'req-f0196be6-8d59-4487-815f-95f23ed59dcf'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.023463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] The token cache is being invalidate because project dc5d85e162a84e4aa383f6c3bcb985d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:53.023749 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.024394 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.025984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1335/2654] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:52 2022] DELETE /identity/v3/projects/dc5d85e162a84e4aa383f6c3bcb985d5 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:53.029804 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:53.031159 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:53.094733 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0196be6-8d59-4487-815f-95f23ed59dcf admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:53.099262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:53.100156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:53.100420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:53.100696 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] PATH_INFO: `/v3/users/74b45956d6484784b32f9467615f804a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:53.101886 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b45956d6484784b32f9467615f804a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:53.104053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:53.145923 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a193f92de13744a0a4b8cd4e9242f403'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.162097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1320/2655] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:52 2022] DELETE /identity/v3/users/a193f92de13744a0a4b8cd4e9242f403 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:53.163087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:53.165261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:53.234940 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d58223-33d9-4a9a-a34e-9369682d6391 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:53.240485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:53.241419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:53.241715 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:53.250127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] PATH_INFO: `/v3/projects/5ad6ec15e5214485944d94f74c7c0704` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:53.250127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad6ec15e5214485944d94f74c7c0704)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:53.250127 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:53.398894 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b45956d6484784b32f9467615f804a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.399069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad6ec15e5214485944d94f74c7c0704', 'request_id': 'req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.399543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b45956d6484784b32f9467615f804a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.406920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b45956d6484784b32f9467615f804a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.545699 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad6ec15e5214485944d94f74c7c0704', 'request_id': 'req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.552182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad6ec15e5214485944d94f74c7c0704', 'request_id': 'req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.564273 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] The token cache is being invalidate because project 5ad6ec15e5214485944d94f74c7c0704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:53.564560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.565216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5e479aff-c6b6-4cf2-a995-09b6cd285bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.566950 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1321/2656] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:53 2022] DELETE /identity/v3/projects/5ad6ec15e5214485944d94f74c7c0704 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:53.568218 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:53.568501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:53.568811 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:53.585622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:53.586863 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:53.645270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1322/2657] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:53.646047 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:53.647342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:53.659028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65a538-62e8-483d-8675-bee3406a3b75 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:53.666450 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b45956d6484784b32f9467615f804a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.666961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:53.666961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:53.666961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:53.666961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] PATH_INFO: `/v3/users/2ccfd9270fc041bfbceb92a32846e789` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:53.667420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ccfd9270fc041bfbceb92a32846e789)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:53.669049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:53.674197 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b45956d6484784b32f9467615f804a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:53.680216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1336/2658] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:53 2022] DELETE /identity/v3/users/74b45956d6484784b32f9467615f804a => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:53.683871 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:53.685085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:53.743457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00411a-5b75-4c7e-bc7e-593b4eb62db7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:53.747570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:53.748359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:53.748718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:53.749067 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:53.815975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:53.817497 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:53.827028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1337/2659] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:54.086325 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ccfd9270fc041bfbceb92a32846e789'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.087104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ccfd9270fc041bfbceb92a32846e789'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.087823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ccfd9270fc041bfbceb92a32846e789'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.100085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ccfd9270fc041bfbceb92a32846e789'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.207414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:54.207895 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:54.221742 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900b727-e1ab-477d-94e6-e2ac99ec32be service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:54.225487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ccfd9270fc041bfbceb92a32846e789'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.231006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:54.233296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:54.233296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:54.235762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] PATH_INFO: `/v3/projects/09c507a0f3dc41a5a0caabdf3fc4057a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:54.235762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c507a0f3dc41a5a0caabdf3fc4057a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:54.236929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:54.245200 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1323/2660] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:53 2022] DELETE /identity/v3/users/2ccfd9270fc041bfbceb92a32846e789 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:54.375446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c507a0f3dc41a5a0caabdf3fc4057a', 'request_id': 'req-af55d97a-23d6-4448-a0c4-7d7d40221cfe'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.381803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c507a0f3dc41a5a0caabdf3fc4057a', 'request_id': 'req-af55d97a-23d6-4448-a0c4-7d7d40221cfe'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.473570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:54.475037 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:54.489241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c507a0f3dc41a5a0caabdf3fc4057a', 'request_id': 'req-af55d97a-23d6-4448-a0c4-7d7d40221cfe'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.502295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] The token cache is being invalidate because project 09c507a0f3dc41a5a0caabdf3fc4057a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:54.502589 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.503256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af55d97a-23d6-4448-a0c4-7d7d40221cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.508082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1338/2661] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:54 2022] DELETE /identity/v3/projects/09c507a0f3dc41a5a0caabdf3fc4057a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:54.512411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e997fb6-61f8-43c1-85be-fee3e2fb8159 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:54.512848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e997fb6-61f8-43c1-85be-fee3e2fb8159 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:54.512951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e997fb6-61f8-43c1-85be-fee3e2fb8159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:54.542047 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-5e997fb6-61f8-43c1-85be-fee3e2fb8159 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:54.547302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-5e997fb6-61f8-43c1-85be-fee3e2fb8159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:54.563180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:54.567491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14465a-7411-47ed-8225-9fd60100bc43 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:54.632320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1339/2662] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:54.642423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:54.642423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:54.642423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:54.642423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] PATH_INFO: `/v3/projects/423a4b3ce644466fb21289009a186292` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:54.642993 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423a4b3ce644466fb21289009a186292)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:54.644158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:54.653343 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7a08d-8d65-4023-b8e2-ffbbc17f7dbb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:54.653638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7a08d-8d65-4023-b8e2-ffbbc17f7dbb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:54.654268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7a08d-8d65-4023-b8e2-ffbbc17f7dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:54.676356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-16b7a08d-8d65-4023-b8e2-ffbbc17f7dbb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:54.677650 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-16b7a08d-8d65-4023-b8e2-ffbbc17f7dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:54.686399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1340/2663] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:54.690474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7a08d-8d65-4023-b8e2-ffbbc17f7dbb None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:54.700182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7a08d-8d65-4023-b8e2-ffbbc17f7dbb None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:54.704881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:54.705538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:54.705821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:54.706099 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:54.706868 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:54.707182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:54.708786 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:54.724760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1341/2664] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:04:54.729152 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:54.738962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd2331-3c21-457d-b324-1f2c7e392a3d admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:54.744426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:54.745062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:54.745206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:54.745491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:54.746194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:54.748107 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:54.848757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423a4b3ce644466fb21289009a186292', 'request_id': 'req-3d4333ca-43ec-4027-9586-27a81e0ce2c3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:54.925726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1342/2665] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:54 2022] POST /identity/v3/projects => generated 407 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:54.927917 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:54.928440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:54.995637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e72c70-45f4-4ca6-9d9b-79279f1070d9 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:54.996942 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423a4b3ce644466fb21289009a186292', 'request_id': 'req-3d4333ca-43ec-4027-9586-27a81e0ce2c3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.000383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:55.001246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:55.001529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:55.001841 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] PATH_INFO: `/v3/projects/1e2d9a2c7328491c88cb60dc541f5fdc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:55.003156 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2d9a2c7328491c88cb60dc541f5fdc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:55.003274 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423a4b3ce644466fb21289009a186292', 'request_id': 'req-3d4333ca-43ec-4027-9586-27a81e0ce2c3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.005081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:55.015056 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] The token cache is being invalidate because project 423a4b3ce644466fb21289009a186292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:55.015345 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.016032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.017587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1324/2666] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:54 2022] DELETE /identity/v3/projects/423a4b3ce644466fb21289009a186292 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:55.021341 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:55.022709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:55.065863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:55.084891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4333ca-43ec-4027-9586-27a81e0ce2c3 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:55.089719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:55.090410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:55.090697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:55.090972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:55.091657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:55.093678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:55.156199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2d9a2c7328491c88cb60dc541f5fdc', 'request_id': 'req-e90687fb-0989-4f72-a18c-347a64c2250a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.162235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2d9a2c7328491c88cb60dc541f5fdc', 'request_id': 'req-e90687fb-0989-4f72-a18c-347a64c2250a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.284447 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1325/2667] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:55 2022] POST /identity/v3/users => generated 304 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:55.285290 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:55.288542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2d9a2c7328491c88cb60dc541f5fdc', 'request_id': 'req-e90687fb-0989-4f72-a18c-347a64c2250a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.290472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:55.300806 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] The token cache is being invalidate because project 1e2d9a2c7328491c88cb60dc541f5fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:55.301248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.301902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.303484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1343/2668] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:54 2022] DELETE /identity/v3/projects/1e2d9a2c7328491c88cb60dc541f5fdc => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:55.303601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296db9ce-7b68-467b-a57f-db91d443bd49 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:55.304235 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:55.305515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:55.361442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:55.362358 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:55.362644 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:55.362922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] PATH_INFO: `/v3/users/bfad769a1eca41b79253e6e413de56d2` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:55.364187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfad769a1eca41b79253e6e413de56d2)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:55.366213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:55.371495 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90687fb-0989-4f72-a18c-347a64c2250a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:55.380757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:55.380757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:55.380757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:55.380757 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:55.381195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:55.381195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:55.390306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1344/2669] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:55.394165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:55.395869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:55.409266 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688c88c-98c1-46e5-9f8b-5b1ef650f262 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:55.413925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:55.421182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:55.421182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:55.421182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] PATH_INFO: `/v3/users/e2b02caafb5d401280fc667143dafe65` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:55.421182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b02caafb5d401280fc667143dafe65)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:55.421182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:55.663363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfad769a1eca41b79253e6e413de56d2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.665087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfad769a1eca41b79253e6e413de56d2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.666440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfad769a1eca41b79253e6e413de56d2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.678909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfad769a1eca41b79253e6e413de56d2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.680574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b02caafb5d401280fc667143dafe65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.681961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b02caafb5d401280fc667143dafe65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.688796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b02caafb5d401280fc667143dafe65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.807902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfad769a1eca41b79253e6e413de56d2'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.821855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1326/2670] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:55 2022] DELETE /identity/v3/users/bfad769a1eca41b79253e6e413de56d2 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:55.822691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:55.824042 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:55.871958 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b02caafb5d401280fc667143dafe65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.885219 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b02caafb5d401280fc667143dafe65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:55.891776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1345/2671] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:04:55 2022] DELETE /identity/v3/users/e2b02caafb5d401280fc667143dafe65 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:55.891985 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d40f4-dd35-4116-8109-2cb835b72bc7 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:55.955489 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38020d58-669b-4397-9b31-48848aedec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:55.956577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38020d58-669b-4397-9b31-48848aedec50 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:55.956577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38020d58-669b-4397-9b31-48848aedec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:55.956800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38020d58-669b-4397-9b31-48848aedec50 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:55.958103 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38020d58-669b-4397-9b31-48848aedec50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:55.960829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38020d58-669b-4397-9b31-48848aedec50 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:55.973948 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1327/2672] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:55.978370 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:55.981082 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:55.995056 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d28a9-c59b-49ab-8d04-3ed45b6f15bb admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:56.000463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:56.001369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.001655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.001935 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] PATH_INFO: `/v3/projects/4672686f256a4de5a81ede8da330b82b/users/d0391be3bde34755b0da3e524a6db360/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.018137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4672686f256a4de5a81ede8da330b82b, user_id=d0391be3bde34755b0da3e524a6db360, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:56.022839 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:56.100324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1346/2673] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:55 2022] PUT /identity/v3/projects/4672686f256a4de5a81ede8da330b82b/users/d0391be3bde34755b0da3e524a6db360/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:56.112650 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.113288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.113578 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.143349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] MFA Rules not processed for user `d0391be3bde34755b0da3e524a6db360`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:56.144658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:56.210888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1328/2674] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:56.216150 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:56.279379 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151dcdf-9964-49a5-be15-944290f9a8ab admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:56.284248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:56.284911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.285203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.285479 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.286210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:56.288324 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:56.351774 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:56.391694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:56.396782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:56.397146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1347/2675] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:56 2022] POST /identity/v3/projects => generated 409 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:56.397146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:56.410070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb926c1c-1f5d-4b2f-8214-1f16c0e0e0f6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:56.481443 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:56.482346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.482634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.482919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] PATH_INFO: `/v3/projects/e9f6b6f98e5d4bc5884d70bd48aaff45` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.484295 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f6b6f98e5d4bc5884d70bd48aaff45)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:56.484567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c048858-b79a-4bb8-a327-f7dccda305f7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:56.486289 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:56.489196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:56.489860 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.490124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.490431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.491147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:56.493085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:56.643214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1348/2676] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:56 2022] POST /identity/v3/users => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:56.644228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:56.646044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:56.662681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e45a9-b068-48bc-882b-cbfecc7303fa admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:56.680710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f6b6f98e5d4bc5884d70bd48aaff45', 'request_id': 'req-9986bc20-4686-44cd-8b7e-cde0114c9e78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:56.725883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:56.726818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.727105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.727384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] PATH_INFO: `/v3/projects/87112d0bbbee45cbbdd077817799dfab` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.728695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87112d0bbbee45cbbdd077817799dfab)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:56.730803 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:56.802658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f6b6f98e5d4bc5884d70bd48aaff45', 'request_id': 'req-9986bc20-4686-44cd-8b7e-cde0114c9e78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:56.821019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f6b6f98e5d4bc5884d70bd48aaff45', 'request_id': 'req-9986bc20-4686-44cd-8b7e-cde0114c9e78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:56.821446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] The token cache is being invalidate because project e9f6b6f98e5d4bc5884d70bd48aaff45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:56.821446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:56.822019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:56.824463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1329/2677] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:56 2022] DELETE /identity/v3/projects/e9f6b6f98e5d4bc5884d70bd48aaff45 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:56.830057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:56.831172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:56.844375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986bc20-4686-44cd-8b7e-cde0114c9e78 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:56.848929 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5eb9a-ef6e-4f17-b363-f90bdef33981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:56.849585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5eb9a-ef6e-4f17-b363-f90bdef33981 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.849865 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5eb9a-ef6e-4f17-b363-f90bdef33981 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.850137 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5eb9a-ef6e-4f17-b363-f90bdef33981 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.850887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5eb9a-ef6e-4f17-b363-f90bdef33981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:56.852561 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5eb9a-ef6e-4f17-b363-f90bdef33981 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:56.862449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1330/2678] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:56.863747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8040-837d-4e1d-85f4-b3c43ab198f6 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.864035 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8040-837d-4e1d-85f4-b3c43ab198f6 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.864308 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8040-837d-4e1d-85f4-b3c43ab198f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.882415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-1c4a8040-837d-4e1d-85f4-b3c43ab198f6 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:56.883603 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1c4a8040-837d-4e1d-85f4-b3c43ab198f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:56.892157 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1331/2679] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:56.892878 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a8040-837d-4e1d-85f4-b3c43ab198f6 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:56.902464 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a8040-837d-4e1d-85f4-b3c43ab198f6 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:56.906947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c2bc9-b41c-4563-a301-a4475f15ea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:56.907812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c2bc9-b41c-4563-a301-a4475f15ea74 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:56.908101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c2bc9-b41c-4563-a301-a4475f15ea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:56.908376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c2bc9-b41c-4563-a301-a4475f15ea74 admin admin] PATH_INFO: `/v3/projects/cd032be8dfbe4f7aabed4de945fc46f5/users/fb1f926c4f0241b9aa1526c22e309aa5/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:56.922839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c2bc9-b41c-4563-a301-a4475f15ea74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd032be8dfbe4f7aabed4de945fc46f5, user_id=fb1f926c4f0241b9aa1526c22e309aa5, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:56.925498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c2bc9-b41c-4563-a301-a4475f15ea74 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.082874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1332/2680] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:56 2022] PUT /identity/v3/projects/cd032be8dfbe4f7aabed4de945fc46f5/users/fb1f926c4f0241b9aa1526c22e309aa5/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:57.084177 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6366-55bd-4b5f-b000-b573073fd345 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.084473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6366-55bd-4b5f-b000-b573073fd345 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.084888 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce6366-55bd-4b5f-b000-b573073fd345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.091199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87112d0bbbee45cbbdd077817799dfab', 'request_id': 'req-9c2950a2-3865-4bb8-bc91-440b110c0155'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:57.098220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87112d0bbbee45cbbdd077817799dfab', 'request_id': 'req-9c2950a2-3865-4bb8-bc91-440b110c0155'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:57.102330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-07ce6366-55bd-4b5f-b000-b573073fd345 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:57.104066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-07ce6366-55bd-4b5f-b000-b573073fd345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:57.163128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1333/2681] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:57.163896 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce6366-55bd-4b5f-b000-b573073fd345 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.173420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce6366-55bd-4b5f-b000-b573073fd345 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.178061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.178236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87112d0bbbee45cbbdd077817799dfab', 'request_id': 'req-9c2950a2-3865-4bb8-bc91-440b110c0155'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:57.178744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.179031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.179307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.180033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.181731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.190727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] The token cache is being invalidate because project 87112d0bbbee45cbbdd077817799dfab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:04:57.191144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:57.191788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:04:57.192484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1334/2682] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:57.196506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.196665 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1349/2683] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:04:56 2022] DELETE /identity/v3/projects/87112d0bbbee45cbbdd077817799dfab => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:57.196791 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:57.196976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.198652 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:57.208173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d0223-2112-4a6d-b612-f09a0b550659 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.210953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2950a2-3865-4bb8-bc91-440b110c0155 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.213120 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52391aaf-9159-416a-a91f-206b64ffc633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.213806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391aaf-9159-416a-a91f-206b64ffc633 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.214115 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391aaf-9159-416a-a91f-206b64ffc633 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.214496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391aaf-9159-416a-a91f-206b64ffc633 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.215221 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391aaf-9159-416a-a91f-206b64ffc633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:57.215510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.215637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391aaf-9159-416a-a91f-206b64ffc633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.216391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.216674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.216955 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] PATH_INFO: `/v3/projects/cd032be8dfbe4f7aabed4de945fc46f5/users/fb1f926c4f0241b9aa1526c22e309aa5/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.217131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391aaf-9159-416a-a91f-206b64ffc633 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.219385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd032be8dfbe4f7aabed4de945fc46f5, user_id=fb1f926c4f0241b9aa1526c22e309aa5, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.222023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.231553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1335/2684] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:04:57.232900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78a0b5-5199-402c-9063-b9bfb8914414 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.233186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78a0b5-5199-402c-9063-b9bfb8914414 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.233461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78a0b5-5199-402c-9063-b9bfb8914414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.237143 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1350/2685] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:57 2022] PUT /identity/v3/projects/cd032be8dfbe4f7aabed4de945fc46f5/users/fb1f926c4f0241b9aa1526c22e309aa5/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:57.237926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.252890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-dd78a0b5-5199-402c-9063-b9bfb8914414 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:57.254133 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-dd78a0b5-5199-402c-9063-b9bfb8914414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:57.298923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fb862-e78c-4243-be6f-d18b8ee60739 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.303540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.304234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.304520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.304801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.305489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.308113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.313496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1336/2686] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:57.314772 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d780-a82b-4a16-af5c-2273e00eec38 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.315061 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d780-a82b-4a16-af5c-2273e00eec38 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.315337 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d780-a82b-4a16-af5c-2273e00eec38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.328710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:04:57.342925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-3b46d780-a82b-4a16-af5c-2273e00eec38 None None] MFA Rules not processed for user `fb1f926c4f0241b9aa1526c22e309aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:57.344370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3b46d780-a82b-4a16-af5c-2273e00eec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:57.408674 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1337/2687] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:57.410097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.410434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.410691 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.429453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:57.430738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:57.439299 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1338/2688] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:57.440054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.441305 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:57.450643 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1351/2689] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:57.451485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.499570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e240667-c194-40a7-9c06-471b58dced72 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.515626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d67469-02e9-4a94-b83e-f96bf867168b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.520564 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.521227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.521508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.521780 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.522532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:04:57.522842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.524444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.537053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1352/2690] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:04:57.537867 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.550550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071a8bc-8b83-4fbe-acc7-191700a7a1fb admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.552312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.552983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.553270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.553671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.555229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.555993 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.556339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.556592 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.557420 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.559350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.616786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.618271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.628093 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1339/2691] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:57.628988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.638974 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bcdba-fc83-46f8-887f-7d25fdcd3354 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.643803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.644453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.644849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.645151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.645928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.647876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.793754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1353/2692] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/users => generated 287 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:57.797898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.807697 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec6b7d-0fe3-498c-816d-b95d6d94d927 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.812243 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.812883 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.813166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.813442 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.814311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.815896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.825836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1354/2693] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:57.833776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.843832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7675a9-6510-47d6-8d97-fe6163a96792 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.848835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.849349 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.849634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.849908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.850660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.852222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.863120 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1355/2694] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:57.867076 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.877234 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6995e-56cf-4d8b-a754-ab0cf7f2f5a7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.880203 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1340/2695] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/projects => generated 369 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:57.882089 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5497426-8e63-4193-a4f9-2b9526fff724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.885460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497426-8e63-4193-a4f9-2b9526fff724 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.885460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497426-8e63-4193-a4f9-2b9526fff724 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.885460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497426-8e63-4193-a4f9-2b9526fff724 admin admin] PATH_INFO: `/v3/projects/a5c6a78958cb4d34862e51557f13d0b1/users/f1314d2b3b8145bbb13aea98702d9b43/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.887022 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:57.901874 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5497426-8e63-4193-a4f9-2b9526fff724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c6a78958cb4d34862e51557f13d0b1, user_id=f1314d2b3b8145bbb13aea98702d9b43, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.904615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5497426-8e63-4193-a4f9-2b9526fff724 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:57.957020 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25cd1-d038-4a11-bae9-6e0a708a8cea admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:57.962270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:57.963550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:57.963837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:57.964117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:57.964813 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:57.966797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.059899 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1356/2696] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:57 2022] PUT /identity/v3/projects/a5c6a78958cb4d34862e51557f13d0b1/users/f1314d2b3b8145bbb13aea98702d9b43/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:58.064961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.065248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.065527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.071653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1341/2697] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:57 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:58.075976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.083572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:58.084802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:58.085462 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75b1a5-fe59-4104-ba5a-8a65e2fbcc54 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.141785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.142549 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.142770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.143051 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.143774 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.144943 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1357/2698] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:58.145490 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.155565 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.156919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1342/2699] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:58.158472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:58.160634 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.170970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cc862-7a55-4aad-8672-df3aea80eada admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.175575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.176232 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.176522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.176800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.177522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.179182 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.199826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1343/2700] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:58.202920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.218150 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f32efc-f73e-4a14-bb4a-c92935d7a193 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.222191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2d63-0710-43ac-83b7-04668b1a9648 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.223593 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487019c6-a9fe-40c4-a436-1f16904e5457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.224488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487019c6-a9fe-40c4-a436-1f16904e5457 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.224775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487019c6-a9fe-40c4-a436-1f16904e5457 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.225054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487019c6-a9fe-40c4-a436-1f16904e5457 admin admin] PATH_INFO: `/v3/projects/b6b733ef712842fe8a9db8c6d88f5603/users/7db2f8cf549c43ed87b459204fc67d78/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.225835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.226508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.226792 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.227063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.236427 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.237961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.240099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487019c6-a9fe-40c4-a436-1f16904e5457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b733ef712842fe8a9db8c6d88f5603, user_id=7db2f8cf549c43ed87b459204fc67d78, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.242817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487019c6-a9fe-40c4-a436-1f16904e5457 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.248452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1358/2701] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:04:58.249326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.303035 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b41a8-b490-4589-a45c-e87c5c869c8b service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.307976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.308677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.308914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.309193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.311030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:58.346198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1344/2702] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:58 2022] PUT /identity/v3/projects/b6b733ef712842fe8a9db8c6d88f5603/users/7db2f8cf549c43ed87b459204fc67d78/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:04:58.357064 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32fc54-b746-4e5f-9904-c925769ccd6e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.357522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32fc54-b746-4e5f-9904-c925769ccd6e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.358039 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32fc54-b746-4e5f-9904-c925769ccd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.375063 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.376526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.387738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ba32fc54-b746-4e5f-9904-c925769ccd6e None None] MFA Rules not processed for user `7db2f8cf549c43ed87b459204fc67d78`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:58.388289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1359/2703] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:58 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:58.388962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ba32fc54-b746-4e5f-9904-c925769ccd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:58.458353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1345/2704] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:04:58.462891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.475144 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab409a-3861-4d68-890b-b2b4d53634bc service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.539738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.540505 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.540857 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.541328 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.542074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.544057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.655885 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1360/2705] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:04:58 2022] POST /identity/v3/projects => generated 369 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:04:58.659921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32fc54-b746-4e5f-9904-c925769ccd6e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.671275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32fc54-b746-4e5f-9904-c925769ccd6e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.726365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.726960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.727249 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.727523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.728223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.730839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.861204 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1346/2706] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:04:58 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:04:58.865167 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.876318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e43b4-56ab-4807-9ca0-769f6cacbcca admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.880818 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.881484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.881768 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.882044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.882834 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.884436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:58.895646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1361/2707] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:04:58.899744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:58.910110 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ae370-29da-4c67-96c6-56766dba63c1 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:58.914748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:58.915640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:58.916495 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:58.916964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] PATH_INFO: `/v3/projects/b8487ae322ed4f70b174b6e7e069cd63/users/4d8f650ac77b43d4b861e03f5f1ead65/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:58.933721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8487ae322ed4f70b174b6e7e069cd63, user_id=4d8f650ac77b43d4b861e03f5f1ead65, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:58.936718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:59.054607 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1347/2708] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:58 2022] PUT /identity/v3/projects/b8487ae322ed4f70b174b6e7e069cd63/users/4d8f650ac77b43d4b861e03f5f1ead65/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:59.065255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:59.069722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1653cd-58ad-4b8b-beee-e67fe76a3cdd admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:59.127400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c062b6-4a13-4c20-aafc-d2a83bf3ccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:59.128145 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c062b6-4a13-4c20-aafc-d2a83bf3ccfc admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:59.128345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c062b6-4a13-4c20-aafc-d2a83bf3ccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:59.128622 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c062b6-4a13-4c20-aafc-d2a83bf3ccfc admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:59.129345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c062b6-4a13-4c20-aafc-d2a83bf3ccfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:59.131165 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c062b6-4a13-4c20-aafc-d2a83bf3ccfc admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:59.142776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1362/2709] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:04:59.147492 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:59.157728 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64bb99-434b-41fb-bf34-3027c4e04e2b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:59.162517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:59.163420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:59.163705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:59.163986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] PATH_INFO: `/v3/projects/b8487ae322ed4f70b174b6e7e069cd63/users/4d8f650ac77b43d4b861e03f5f1ead65/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:59.166158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8487ae322ed4f70b174b6e7e069cd63, user_id=4d8f650ac77b43d4b861e03f5f1ead65, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:59.168867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:59.180327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1348/2710] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:04:59 2022] PUT /identity/v3/projects/b8487ae322ed4f70b174b6e7e069cd63/users/4d8f650ac77b43d4b861e03f5f1ead65/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:04:59.192172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0831e-6c1c-4f40-ace3-ed3c11a69612 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:59.197246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0831e-6c1c-4f40-ace3-ed3c11a69612 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:59.197246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0831e-6c1c-4f40-ace3-ed3c11a69612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:59.221737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e3b0831e-6c1c-4f40-ace3-ed3c11a69612 None None] MFA Rules not processed for user `4d8f650ac77b43d4b861e03f5f1ead65`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:04:59.223171 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e3b0831e-6c1c-4f40-ace3-ed3c11a69612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:59.295147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1363/2711] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:04:59 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:04:59.307386 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:59.309491 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:04:59.370353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef36a15-2d35-413e-99d7-4e8174bfde9e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:59.374262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa341d-d5d4-4090-bea4-41f4fad225bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:59.374915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa341d-d5d4-4090-bea4-41f4fad225bb service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:59.375201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa341d-d5d4-4090-bea4-41f4fad225bb service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:59.375478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa341d-d5d4-4090-bea4-41f4fad225bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:59.384917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa341d-d5d4-4090-bea4-41f4fad225bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:59.386398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa341d-d5d4-4090-bea4-41f4fad225bb service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:59.395762 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1349/2712] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:04:59.583207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0831e-6c1c-4f40-ace3-ed3c11a69612 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:04:59.592937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0831e-6c1c-4f40-ace3-ed3c11a69612 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:04:59.596775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:04:59.597576 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:04:59.597862 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:04:59.598140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:04:59.655508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:04:59.656970 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:04:59.666691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1364/2713] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:00.007574 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:00.021195 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc5e9-9621-471b-97c2-385570fe3473 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:00.021703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc5e9-9621-471b-97c2-385570fe3473 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:00.021815 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc5e9-9621-471b-97c2-385570fe3473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:00.049500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-94bfc5e9-9621-471b-97c2-385570fe3473 None None] MFA Rules not processed for user `f1314d2b3b8145bbb13aea98702d9b43`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:00.050735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-94bfc5e9-9621-471b-97c2-385570fe3473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:00.123223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1350/2714] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:00.134226 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:00.143782 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab15573-acf3-4bfe-949a-2405a668a2aa service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:00.147368 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a2c09-93f6-4993-8b03-db8dd6a596cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:00.148008 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2c09-93f6-4993-8b03-db8dd6a596cf service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:00.148290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2c09-93f6-4993-8b03-db8dd6a596cf service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:00.148563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a2c09-93f6-4993-8b03-db8dd6a596cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:00.157854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a2c09-93f6-4993-8b03-db8dd6a596cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:00.160706 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a2c09-93f6-4993-8b03-db8dd6a596cf service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:00.168385 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1365/2715] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:01.157153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:02.299728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:03.505720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:03.544460 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:04.576657 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:04.966706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfc5e9-9621-471b-97c2-385570fe3473 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:05.030485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfc5e9-9621-471b-97c2-385570fe3473 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:05.035167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:05.035983 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:05.036296 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:05.036572 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] PATH_INFO: `/v3/users/7db2f8cf549c43ed87b459204fc67d78` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:05.037972 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db2f8cf549c43ed87b459204fc67d78)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:05.039966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:05.256277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db2f8cf549c43ed87b459204fc67d78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:05.256922 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db2f8cf549c43ed87b459204fc67d78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:05.257515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db2f8cf549c43ed87b459204fc67d78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:05.269355 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db2f8cf549c43ed87b459204fc67d78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:05.281453 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:05.396764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-329ee0a2-a4ef-473e-8d63-282bc9b9b0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db2f8cf549c43ed87b459204fc67d78'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:05.409444 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1351/2716] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:04 2022] DELETE /identity/v3/users/7db2f8cf549c43ed87b459204fc67d78 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:05.413896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:05.414222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:05.414510 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:05.432891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:05.434088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:05.492625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:05.492939 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:05.493267 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:05.493365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1366/2717] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:05.508765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:05.510490 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:05.511619 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] MFA Rules not processed for user `f1314d2b3b8145bbb13aea98702d9b43`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:05.512735 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:05.569193 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f0a51-644e-45b6-8097-ab90878ffc68 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:05.571388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1352/2718] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:05.576237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:05.576237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:05.576237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:05.576237 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:05.586638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:05.586638 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:05.597310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1367/2719] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:06.201601 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:06.319584 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:06.321262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:06.349727 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f87c7-78f5-472b-b29e-985002ba3a00 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:06.355880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:06.357272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:06.357839 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:06.358240 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] PATH_INFO: `/v3/projects/b6b733ef712842fe8a9db8c6d88f5603` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:06.359737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b733ef712842fe8a9db8c6d88f5603)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:06.362293 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:06.466809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b733ef712842fe8a9db8c6d88f5603', 'request_id': 'req-d30f1102-9db8-48a3-9acc-b43cd0287f8e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.574610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b733ef712842fe8a9db8c6d88f5603', 'request_id': 'req-d30f1102-9db8-48a3-9acc-b43cd0287f8e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.580845 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b733ef712842fe8a9db8c6d88f5603', 'request_id': 'req-d30f1102-9db8-48a3-9acc-b43cd0287f8e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.592271 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] The token cache is being invalidate because project b6b733ef712842fe8a9db8c6d88f5603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:06.592558 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.593194 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.595070 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1353/2720] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:06 2022] DELETE /identity/v3/projects/b6b733ef712842fe8a9db8c6d88f5603 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:06.599131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:06.610072 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6033d-d09a-4202-89e4-1bbbab665f42 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:06.667634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:06.668514 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:06.668800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:06.669080 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] PATH_INFO: `/v3/users/4d8f650ac77b43d4b861e03f5f1ead65` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:06.670286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8f650ac77b43d4b861e03f5f1ead65)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:06.672272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:06.842817 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:06.857318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8f650ac77b43d4b861e03f5f1ead65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.857961 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8f650ac77b43d4b861e03f5f1ead65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.864776 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8f650ac77b43d4b861e03f5f1ead65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.984766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8f650ac77b43d4b861e03f5f1ead65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:06.997677 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-4f382fde-5af9-49a9-9496-e7f80a328ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8f650ac77b43d4b861e03f5f1ead65'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:07.007060 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1368/2721] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:06 2022] DELETE /identity/v3/users/4d8f650ac77b43d4b861e03f5f1ead65 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:07.439095 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:07.442906 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:07.466434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1102-9db8-48a3-9acc-b43cd0287f8e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:07.563241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:07.564237 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:07.564631 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:07.565014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] PATH_INFO: `/v3/projects/b8487ae322ed4f70b174b6e7e069cd63` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:07.566568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8487ae322ed4f70b174b6e7e069cd63)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:07.568988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:07.677346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:07.794215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8487ae322ed4f70b174b6e7e069cd63', 'request_id': 'req-d48a2468-aad0-493d-8663-05f3c3375de1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:07.956485 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8487ae322ed4f70b174b6e7e069cd63', 'request_id': 'req-d48a2468-aad0-493d-8663-05f3c3375de1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:07.963885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8487ae322ed4f70b174b6e7e069cd63', 'request_id': 'req-d48a2468-aad0-493d-8663-05f3c3375de1'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:07.975532 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] The token cache is being invalidate because project b8487ae322ed4f70b174b6e7e069cd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:07.975925 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:07.976596 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d48a2468-aad0-493d-8663-05f3c3375de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:07.978388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1354/2722] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:07 2022] DELETE /identity/v3/projects/b8487ae322ed4f70b174b6e7e069cd63 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:07.983529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f873f-a36c-4bc5-a6ff-85d9d02d6517 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:07.987630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f873f-a36c-4bc5-a6ff-85d9d02d6517 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:07.987630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f873f-a36c-4bc5-a6ff-85d9d02d6517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.003176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-e77f873f-a36c-4bc5-a6ff-85d9d02d6517 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:08.004486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e77f873f-a36c-4bc5-a6ff-85d9d02d6517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:08.070788 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1369/2723] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:08.079287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c4c4f-5de6-46a2-8a0c-411eac2fcf8b None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.079575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c4c4f-5de6-46a2-8a0c-411eac2fcf8b None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.079847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c4c4f-5de6-46a2-8a0c-411eac2fcf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.097749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ba3c4c4f-5de6-46a2-8a0c-411eac2fcf8b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:08.099011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ba3c4c4f-5de6-46a2-8a0c-411eac2fcf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:08.107210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1355/2724] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:08.111557 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f873f-a36c-4bc5-a6ff-85d9d02d6517 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:08.123013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f873f-a36c-4bc5-a6ff-85d9d02d6517 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:08.127983 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:08.131448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.131448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.131448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.131448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:08.131448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:08.131942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:08.144795 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1370/2725] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:05:08.149408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c4c4f-5de6-46a2-8a0c-411eac2fcf8b None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:08.159436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c4c4f-5de6-46a2-8a0c-411eac2fcf8b None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:08.164375 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:08.165030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.165692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.165994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.166866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:08.169361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:08.273173 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1356/2726] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:08 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:08.278416 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:08.288085 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8bb2c-a8c3-422e-a263-e012e100d12b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:08.347533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:08.348210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.348491 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.348767 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.349511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:08.351487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:08.489207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1371/2727] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:08 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:08.493091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:08.506694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b7f85-a815-4b36-897e-f2dd7272a47a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:08.513442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:08.514151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.514585 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.514860 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.515610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:08.517261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:08.530344 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1357/2728] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:08.542290 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:08.567359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:08.567478 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f509ee9-4b3e-4fb5-8155-e0d133c4650b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:08.569292 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc59c8-723c-4127-ae96-d5934c38b6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:08.578870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc59c8-723c-4127-ae96-d5934c38b6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.579239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc59c8-723c-4127-ae96-d5934c38b6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.579526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc59c8-723c-4127-ae96-d5934c38b6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.580285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc59c8-723c-4127-ae96-d5934c38b6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:08.581909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc59c8-723c-4127-ae96-d5934c38b6d0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:08.592879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1372/2729] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:08.597609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:08.608254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b315-e261-4d6b-ab16-462365b4f638 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:08.612834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:08.613738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.613995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.614297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] PATH_INFO: `/v3/projects/727b5eea20044f16971b553331d703cb/users/ca3fc7106f7444378397ff34bbe62e76/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.629467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=727b5eea20044f16971b553331d703cb, user_id=ca3fc7106f7444378397ff34bbe62e76, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:08.632179 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:08.780828 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1358/2730] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:08 2022] PUT /identity/v3/projects/727b5eea20044f16971b553331d703cb/users/ca3fc7106f7444378397ff34bbe62e76/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:08.791407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dedbf-464d-4b85-b474-b8ccfc90abde None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.791921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dedbf-464d-4b85-b474-b8ccfc90abde None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.791921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dedbf-464d-4b85-b474-b8ccfc90abde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.819396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-c06dedbf-464d-4b85-b474-b8ccfc90abde None None] MFA Rules not processed for user `ca3fc7106f7444378397ff34bbe62e76`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:08.820597 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c06dedbf-464d-4b85-b474-b8ccfc90abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:08.887288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1373/2731] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:08.891893 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:08.963841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860df6-3f4d-466a-a6c2-12c89f73f93c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:08.969653 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:08.970483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:08.970773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:08.971048 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:08.972029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:08.973914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.065338 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1359/2732] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:08 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:09.069486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dedbf-464d-4b85-b474-b8ccfc90abde None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:09.084321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dedbf-464d-4b85-b474-b8ccfc90abde None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:09.146701 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:09.147384 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.147669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.147940 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.148664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:09.151413 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.289524 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1374/2733] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:09 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:09.293690 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:09.304275 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10f2a3-a052-4bb1-b9d4-e7b5026a36b7 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:09.309023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:09.309820 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.310219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.310565 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.311363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:09.313079 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.325705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1360/2734] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:09.330965 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:09.343387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa1c7e-ed2c-4ff0-870f-3b4aba1f03b5 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:09.348596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:09.349484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.349765 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.350038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] PATH_INFO: `/v3/projects/70186980eb554192bfd26aa014a83729/users/1830a9a78d8b4819b77f3c71147d3450/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.365600 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70186980eb554192bfd26aa014a83729, user_id=1830a9a78d8b4819b77f3c71147d3450, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:09.368338 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.506406 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1375/2735] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:09 2022] PUT /identity/v3/projects/70186980eb554192bfd26aa014a83729/users/1830a9a78d8b4819b77f3c71147d3450/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:09.511160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:09.521592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8173a-0159-47c6-b5d0-a5adfbd611bf admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:09.595790 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45e695-d623-4035-b9b1-8b96639f55b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:09.596698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45e695-d623-4035-b9b1-8b96639f55b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.597130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45e695-d623-4035-b9b1-8b96639f55b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.597477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45e695-d623-4035-b9b1-8b96639f55b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.598446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45e695-d623-4035-b9b1-8b96639f55b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:09.600512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45e695-d623-4035-b9b1-8b96639f55b8 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.614094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1361/2736] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:09.615358 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:09.628870 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6860b-c52e-453f-98b9-c45a843c1b96 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:09.633929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:09.635754 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.636506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.636506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] PATH_INFO: `/v3/projects/70186980eb554192bfd26aa014a83729/users/1830a9a78d8b4819b77f3c71147d3450/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.638891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70186980eb554192bfd26aa014a83729, user_id=1830a9a78d8b4819b77f3c71147d3450, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:09.641703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.659458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1376/2737] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:09 2022] PUT /identity/v3/projects/70186980eb554192bfd26aa014a83729/users/1830a9a78d8b4819b77f3c71147d3450/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:09.673235 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.673777 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.674335 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.708682 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] MFA Rules not processed for user `1830a9a78d8b4819b77f3c71147d3450`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:09.710069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:09.732163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:09.734022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:09.787793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1362/2738] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:09.800331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:09.801624 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:09.807050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408852b9-dd00-424d-bfcf-cd48201ed857 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:09.812877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:09.818335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.818335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.818335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] PATH_INFO: `/v3/users/d0391be3bde34755b0da3e524a6db360` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.818335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0391be3bde34755b0da3e524a6db360)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:09.818335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.870416 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd953-7082-4c05-954b-5f05b97d67c8 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:09.874044 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37288c45-3601-4bfc-9d62-d9f2ae9a1c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:09.874816 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37288c45-3601-4bfc-9d62-d9f2ae9a1c33 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:09.875107 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37288c45-3601-4bfc-9d62-d9f2ae9a1c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:09.875385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37288c45-3601-4bfc-9d62-d9f2ae9a1c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:09.884763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37288c45-3601-4bfc-9d62-d9f2ae9a1c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:09.887329 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37288c45-3601-4bfc-9d62-d9f2ae9a1c33 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:09.896146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1363/2739] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:05:10.143122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0391be3bde34755b0da3e524a6db360'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:10.149452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0391be3bde34755b0da3e524a6db360'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:10.151345 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0391be3bde34755b0da3e524a6db360'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:10.305474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0391be3bde34755b0da3e524a6db360'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:10.318661 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:10.327695 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0391be3bde34755b0da3e524a6db360'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:10.341433 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1377/2740] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:09 2022] DELETE /identity/v3/users/d0391be3bde34755b0da3e524a6db360 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:10.347656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:10.348318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:10.348765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:10.368326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:10.371131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:10.459977 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1364/2741] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:10.474051 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:10.476172 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:10.557827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09185f-bd8f-457b-b93c-437b6b9baddc admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:10.563508 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:10.564227 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:10.564516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:10.564880 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:10.576499 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:10.586976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:10.594760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:10.606498 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1378/2742] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:05:11.199388 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:11.201500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:11.214270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9443c-ec21-4ea5-9c65-f0cccada839e None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:11.219143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:11.220282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:11.220567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:11.220836 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] PATH_INFO: `/v3/projects/4672686f256a4de5a81ede8da330b82b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:11.222098 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4672686f256a4de5a81ede8da330b82b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:11.231411 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:11.263077 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:11.264469 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:11.334028 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc0de-7b84-4703-8971-4b3003309a1d service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:11.339617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1830a9a78d8b4819b77f3c71147d3450', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70186980eb554192bfd26aa014a83729', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:11.340646 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:11.340950 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:11.341236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:11.342474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:11.343258 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:11.345541 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:11.359869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1379/2743] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:05:11.375723 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:11.382082 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4672686f256a4de5a81ede8da330b82b', 'request_id': 'req-8d908bb8-c510-4e19-92e6-9321c3344ffd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.387784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe6749-be7e-4c85-a333-337dda66e772 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:11.445890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1830a9a78d8b4819b77f3c71147d3450', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70186980eb554192bfd26aa014a83729', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:11.446570 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:11.446929 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:11.448610 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:11.449340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:11.451268 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:11.483637 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4672686f256a4de5a81ede8da330b82b', 'request_id': 'req-8d908bb8-c510-4e19-92e6-9321c3344ffd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.489736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4672686f256a4de5a81ede8da330b82b', 'request_id': 'req-8d908bb8-c510-4e19-92e6-9321c3344ffd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.502677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] The token cache is being invalidate because project 4672686f256a4de5a81ede8da330b82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:11.502961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.503630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.505468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1365/2744] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:11 2022] DELETE /identity/v3/projects/4672686f256a4de5a81ede8da330b82b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:11.509159 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:11.511867 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:11.604049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d908bb8-c510-4e19-92e6-9321c3344ffd admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:11.604049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:11.604049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:11.604049 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:11.604546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] PATH_INFO: `/v3/users/fb1f926c4f0241b9aa1526c22e309aa5` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:11.604546 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1f926c4f0241b9aa1526c22e309aa5)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:11.605215 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:11.609204 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1380/2745] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:11 2022] POST /identity/v3/projects => generated 353 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:11.610501 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:11.612436 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:11.701984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbc0e9-8034-4cf8-a89a-2f7210896d5a tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:11.706655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1830a9a78d8b4819b77f3c71147d3450', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70186980eb554192bfd26aa014a83729', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:11.707329 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:11.708037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:11.708037 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:11.708613 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:11.708920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:11.710552 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:11.726520 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1381/2746] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:05:11.766467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:11.777259 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd08cd5-9d60-4860-8b37-5408b480c883 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:11.782263 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1830a9a78d8b4819b77f3c71147d3450', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70186980eb554192bfd26aa014a83729', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:11.783249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:11.783543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:11.783825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] PATH_INFO: `/v3/projects/37efb06abe004a2192c832c64677279b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:11.785091 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorizing `identity:delete_project(project_id=37efb06abe004a2192c832c64677279b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:11.787118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:11.925760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1f926c4f0241b9aa1526c22e309aa5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.926429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1f926c4f0241b9aa1526c22e309aa5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.927030 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1f926c4f0241b9aa1526c22e309aa5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.939187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1f926c4f0241b9aa1526c22e309aa5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.959815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37efb06abe004a2192c832c64677279b', 'request_id': 'req-7b77346a-b6de-4d05-8d9f-6f20d021efd7'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:11.967579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37efb06abe004a2192c832c64677279b', 'request_id': 'req-7b77346a-b6de-4d05-8d9f-6f20d021efd7'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:12.036812 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1f926c4f0241b9aa1526c22e309aa5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:12.052381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1366/2747] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:11 2022] DELETE /identity/v3/users/fb1f926c4f0241b9aa1526c22e309aa5 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:12.104055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37efb06abe004a2192c832c64677279b', 'request_id': 'req-7b77346a-b6de-4d05-8d9f-6f20d021efd7'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:12.118513 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] The token cache is being invalidate because project 37efb06abe004a2192c832c64677279b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:12.118931 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:12.119703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:12.121715 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1382/2748] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:11 2022] DELETE /identity/v3/projects/37efb06abe004a2192c832c64677279b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:12.374678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:12.785575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:12.789128 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:12.882622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4831458-46a3-4aec-bbba-639a1c1902de admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:12.888666 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:12.889483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:12.889797 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:12.890045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] PATH_INFO: `/v3/projects/cd032be8dfbe4f7aabed4de945fc46f5` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:12.891408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd032be8dfbe4f7aabed4de945fc46f5)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:12.893359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:12.946114 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:12.948728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:13.021340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77346a-b6de-4d05-8d9f-6f20d021efd7 tempest-VolumeQuotaClassesTest-1831028964 tempest-VolumeQuotaClassesTest-1831028964-project-admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.028228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.029487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.030000 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.030502 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] PATH_INFO: `/v3/users/ca3fc7106f7444378397ff34bbe62e76` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.032254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca3fc7106f7444378397ff34bbe62e76)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.035027 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:13.041829 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd032be8dfbe4f7aabed4de945fc46f5', 'request_id': 'req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.223301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd032be8dfbe4f7aabed4de945fc46f5', 'request_id': 'req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.229298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd032be8dfbe4f7aabed4de945fc46f5', 'request_id': 'req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.242630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] The token cache is being invalidate because project cd032be8dfbe4f7aabed4de945fc46f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:13.242970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.245326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-cd5cc0fd-5f42-4940-8880-ad8690e47fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.247213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1367/2749] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:12 2022] DELETE /identity/v3/projects/cd032be8dfbe4f7aabed4de945fc46f5 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:13.252183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2a72a-0a5f-441a-96ef-79664b78ed2c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.252587 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2a72a-0a5f-441a-96ef-79664b78ed2c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.253000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2a72a-0a5f-441a-96ef-79664b78ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.273167 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-45b2a72a-0a5f-441a-96ef-79664b78ed2c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:13.274419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-45b2a72a-0a5f-441a-96ef-79664b78ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:13.334399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1368/2750] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:13.342473 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8e97e-7724-4084-8cd2-4bc3a877866c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.342761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8e97e-7724-4084-8cd2-4bc3a877866c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.343034 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8e97e-7724-4084-8cd2-4bc3a877866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.360583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-e9f8e97e-7724-4084-8cd2-4bc3a877866c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:13.361809 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-e9f8e97e-7724-4084-8cd2-4bc3a877866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:13.370394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1369/2751] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:13.374394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8e97e-7724-4084-8cd2-4bc3a877866c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:13.382953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8e97e-7724-4084-8cd2-4bc3a877866c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.388151 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.388775 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.389062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.389372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.394279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:13.394279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.394279 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:13.403723 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1370/2752] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:05:13.407718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:13.419238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ec37-5de1-49ad-b315-85cb6399d63e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.425050 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.425719 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.426001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.426408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.427099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.429014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:13.430084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca3fc7106f7444378397ff34bbe62e76'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.430791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca3fc7106f7444378397ff34bbe62e76'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.438921 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca3fc7106f7444378397ff34bbe62e76'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.533904 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca3fc7106f7444378397ff34bbe62e76'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.541500 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1371/2753] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:13 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:05:13.545507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:13.551006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:13.551148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-1804c274-8c9c-42c2-b471-df21cf5345cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca3fc7106f7444378397ff34bbe62e76'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:13.557391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1383/2754] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:12 2022] DELETE /identity/v3/users/ca3fc7106f7444378397ff34bbe62e76 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:13.561303 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.561585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.561861 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.580188 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:13.581392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:13.619616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069dd6c-f3aa-4032-8078-cadec5a95421 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.646926 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1384/2755] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:13.657315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:13.659045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:13.684827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.692919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.692919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.692919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.692919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.692919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:13.721845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02e9a9-a3d9-4c51-9a48-362f2f1ed481 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.728725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:13.728725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.728725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.728725 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.729236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.741487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.743560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:13.755511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1385/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:13.805717 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1372/2757] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:13 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:05:13.809319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:13.820492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c51b9-d54e-42af-977c-90b3ba168de2 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.829800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.849902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.849902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.849902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.849902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.849902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:13.849902 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1386/2758] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:13.851292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:13.863509 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fb2e1-5855-4d37-9b8c-c85d1a91584c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.874932 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa6835-7657-4f7d-a1ba-041cbc5329ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.876507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa6835-7657-4f7d-a1ba-041cbc5329ef admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.877192 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa6835-7657-4f7d-a1ba-041cbc5329ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.878516 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa6835-7657-4f7d-a1ba-041cbc5329ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.883408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa6835-7657-4f7d-a1ba-041cbc5329ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.883749 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa6835-7657-4f7d-a1ba-041cbc5329ef admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:13.894995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1373/2759] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:13.899278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:13.909794 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065abb95-ca54-45c2-b189-ef7d4050d607 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:13.916529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:13.916529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:13.916529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:13.916529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] PATH_INFO: `/v3/projects/cce07fd74cb74e39b32d2604aae75a48/users/ef6af6e0e63f46c084618008142106c5/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:13.932522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce07fd74cb74e39b32d2604aae75a48, user_id=ef6af6e0e63f46c084618008142106c5, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:13.935249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:14.042062 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1387/2760] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:13 2022] PUT /identity/v3/projects/cce07fd74cb74e39b32d2604aae75a48/users/ef6af6e0e63f46c084618008142106c5/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:14.054709 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f10c-fb79-44b5-b8e2-f97531bca767 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:14.055099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f10c-fb79-44b5-b8e2-f97531bca767 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:14.055445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f10c-fb79-44b5-b8e2-f97531bca767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:14.083757 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-38f9f10c-fb79-44b5-b8e2-f97531bca767 None None] MFA Rules not processed for user `ef6af6e0e63f46c084618008142106c5`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:14.084973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-38f9f10c-fb79-44b5-b8e2-f97531bca767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:14.153557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1374/2761] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:14.163781 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:14.166415 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:14.223688 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6bc00-96bc-4d11-9882-64412e0a6c6b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:14.227365 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:14.228593 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:14.228873 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:14.229147 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:14.237956 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:14.239485 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:14.249537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1388/2762] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:14.269277 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9f10c-fb79-44b5-b8e2-f97531bca767 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:14.341014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9f10c-fb79-44b5-b8e2-f97531bca767 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:14.346287 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:14.347172 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:14.347450 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:14.347721 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] PATH_INFO: `/v3/users/ef6af6e0e63f46c084618008142106c5` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:14.348928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6af6e0e63f46c084618008142106c5)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:14.351208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:14.470746 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:14.472170 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:14.487308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a79b77-db37-4f42-96fd-9ccea899ca82 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:14.492845 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:14.493976 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:14.494383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:14.494668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] PATH_INFO: `/v3/projects/727b5eea20044f16971b553331d703cb` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:14.496130 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=727b5eea20044f16971b553331d703cb)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:14.498081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:14.594548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6af6e0e63f46c084618008142106c5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.596784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6af6e0e63f46c084618008142106c5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.596784 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6af6e0e63f46c084618008142106c5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.608402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6af6e0e63f46c084618008142106c5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.666948 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727b5eea20044f16971b553331d703cb', 'request_id': 'req-e9a9abe1-e970-4952-9f68-ed10a669ab85'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.673274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727b5eea20044f16971b553331d703cb', 'request_id': 'req-e9a9abe1-e970-4952-9f68-ed10a669ab85'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.739227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d2d00e5b-c041-4ce2-88e5-b1bdcff05594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6af6e0e63f46c084618008142106c5'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.755616 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1375/2763] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:14 2022] DELETE /identity/v3/users/ef6af6e0e63f46c084618008142106c5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:14.761822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:14.762293 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:14.762706 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:14.780337 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:14.781525 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:14.809887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:14.842278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1376/2764] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:14.852629 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:14.853970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:14.859691 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727b5eea20044f16971b553331d703cb', 'request_id': 'req-e9a9abe1-e970-4952-9f68-ed10a669ab85'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.872732 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] The token cache is being invalidate because project 727b5eea20044f16971b553331d703cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:14.873098 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.873863 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:14.875734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1389/2765] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:14 2022] DELETE /identity/v3/projects/727b5eea20044f16971b553331d703cb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:14.879350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:14.883669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:14.913916 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee795720-808c-4e36-b9fe-caa139ccd0e7 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:14.919738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:14.920474 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:14.920831 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:14.921176 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:14.923077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:14.936111 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:14.944573 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:14.948973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1377/2766] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:05:14.959272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9abe1-e970-4952-9f68-ed10a669ab85 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:14.965287 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:14.966199 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:14.966483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:14.966764 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] PATH_INFO: `/v3/users/1830a9a78d8b4819b77f3c71147d3450` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:14.967956 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1830a9a78d8b4819b77f3c71147d3450)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:14.970191 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:15.256670 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1830a9a78d8b4819b77f3c71147d3450'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.257372 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1830a9a78d8b4819b77f3c71147d3450'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.265212 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1830a9a78d8b4819b77f3c71147d3450'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.350708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1830a9a78d8b4819b77f3c71147d3450'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.365532 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-772a23b8-e664-4017-a346-c59a5c4103e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1830a9a78d8b4819b77f3c71147d3450'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.372890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1390/2767] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:14 2022] DELETE /identity/v3/users/1830a9a78d8b4819b77f3c71147d3450 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:15.480027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:15.482503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:15.493748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcc0e2-30cf-44d5-a786-fbcb9358aaf8 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:15.559518 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:15.560395 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:15.560679 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:15.560957 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] PATH_INFO: `/v3/projects/cce07fd74cb74e39b32d2604aae75a48` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:15.562144 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce07fd74cb74e39b32d2604aae75a48)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:15.564163 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:15.642496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:15.693976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce07fd74cb74e39b32d2604aae75a48', 'request_id': 'req-3d1a8691-e549-4c9a-a921-292e69eb5edc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.830763 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce07fd74cb74e39b32d2604aae75a48', 'request_id': 'req-3d1a8691-e549-4c9a-a921-292e69eb5edc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.836592 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce07fd74cb74e39b32d2604aae75a48', 'request_id': 'req-3d1a8691-e549-4c9a-a921-292e69eb5edc'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.849024 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] The token cache is being invalidate because project cce07fd74cb74e39b32d2604aae75a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:15.849300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.849941 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3d1a8691-e549-4c9a-a921-292e69eb5edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:15.851510 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1378/2768] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:15 2022] DELETE /identity/v3/projects/cce07fd74cb74e39b32d2604aae75a48 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:15.856369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:15.856667 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:15.856954 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:15.874981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:15.876184 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:15.936455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1391/2769] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:15.944184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39a15c-b462-4e21-b835-946f40f297c8 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:15.944471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39a15c-b462-4e21-b835-946f40f297c8 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:15.944754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39a15c-b462-4e21-b835-946f40f297c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:15.963006 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9e39a15c-b462-4e21-b835-946f40f297c8 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:15.964361 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9e39a15c-b462-4e21-b835-946f40f297c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:15.975995 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:15.976148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1379/2770] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:15.976959 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39a15c-b462-4e21-b835-946f40f297c8 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:15.977702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:15.987536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39a15c-b462-4e21-b835-946f40f297c8 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:15.992298 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:15.992973 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:15.993261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:15.993540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:15.994314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:15.995454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:15.996963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.001388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c6695-e310-4704-8a67-1e58f441a3da None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.006716 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.012654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.012654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.012654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] PATH_INFO: `/v3/projects/70186980eb554192bfd26aa014a83729` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.012654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=70186980eb554192bfd26aa014a83729)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.012654 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.013140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1380/2771] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:05:16.016234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.026451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1473-3adc-4307-bc81-289927894add admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.031149 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.031826 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.032141 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.032417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.033101 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.035102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.150712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1381/2772] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:05:16.152342 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.166057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70186980eb554192bfd26aa014a83729', 'request_id': 'req-12a36303-5ab6-4ad9-a668-68856b9e38eb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:16.173075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70186980eb554192bfd26aa014a83729', 'request_id': 'req-12a36303-5ab6-4ad9-a668-68856b9e38eb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:16.217548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96167b91-f21d-49a6-a7b5-fea20ad365c1 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.222605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.223283 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.223609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.223899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.224617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.226798 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.275517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70186980eb554192bfd26aa014a83729', 'request_id': 'req-12a36303-5ab6-4ad9-a668-68856b9e38eb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:16.287105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] The token cache is being invalidate because project 70186980eb554192bfd26aa014a83729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:16.287388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:16.288023 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-12a36303-5ab6-4ad9-a668-68856b9e38eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:16.289581 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1392/2773] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:15 2022] DELETE /identity/v3/projects/70186980eb554192bfd26aa014a83729 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:16.295446 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cce85-2ec9-47c9-b125-0c5d219b567d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.295740 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cce85-2ec9-47c9-b125-0c5d219b567d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.296022 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cce85-2ec9-47c9-b125-0c5d219b567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.313933 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-3a9cce85-2ec9-47c9-b125-0c5d219b567d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:16.315176 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3a9cce85-2ec9-47c9-b125-0c5d219b567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:16.323125 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1393/2774] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:16.330588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afece108-6313-482a-b90d-0b03ad03ca1b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.330908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afece108-6313-482a-b90d-0b03ad03ca1b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.331195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afece108-6313-482a-b90d-0b03ad03ca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.349123 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-afece108-6313-482a-b90d-0b03ad03ca1b None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:16.350326 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-afece108-6313-482a-b90d-0b03ad03ca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:16.358942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1394/2775] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:16.364854 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afece108-6313-482a-b90d-0b03ad03ca1b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.373605 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afece108-6313-482a-b90d-0b03ad03ca1b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.378179 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.378824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.379111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.379388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.380101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:16.380412 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.382003 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.393118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1395/2776] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:05:16.397102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.403866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1382/2777] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:05:16.407274 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224e9b4-8dcf-4477-80dc-2db116fe00de admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.407404 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.409059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:16.412149 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.412785 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.413070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.413347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.414160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.416081 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.421952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95ce4e-aa69-4d03-9861-f101737f3242 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.427190 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.427898 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.428180 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.428903 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.430378 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.436980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.443827 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1383/2778] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:05:16.448484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.458403 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3de6fa-f78f-4387-84e6-ab3c856e122d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.462994 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.464748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.470879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.470879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.470879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.470879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.479731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1384/2779] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:05:16.483740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.493737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1846a-71be-4eec-852f-e7ae08b2f11c admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.499234 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912d39d-e741-4ce4-9333-104f357522a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.500104 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d39d-e741-4ce4-9333-104f357522a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.500383 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d39d-e741-4ce4-9333-104f357522a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.500654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d39d-e741-4ce4-9333-104f357522a6 admin admin] PATH_INFO: `/v3/projects/688205eadcc14900913fa218d1bb2889/users/a2d7054acaad49c2beb0bbd0ba5f9e0a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.515786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912d39d-e741-4ce4-9333-104f357522a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688205eadcc14900913fa218d1bb2889, user_id=a2d7054acaad49c2beb0bbd0ba5f9e0a, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.518507 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912d39d-e741-4ce4-9333-104f357522a6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.542240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1396/2780] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:05:16.546110 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.617537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c152d0-b794-4804-8d60-832d90bae9ad admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.622295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.623001 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.623378 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.623744 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.624438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.626430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.643005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1385/2781] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:16 2022] PUT /identity/v3/projects/688205eadcc14900913fa218d1bb2889/users/a2d7054acaad49c2beb0bbd0ba5f9e0a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:16.654131 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.654373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.654654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.682442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] MFA Rules not processed for user `a2d7054acaad49c2beb0bbd0ba5f9e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:16.683822 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:16.711333 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1397/2782] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:16.716074 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.726773 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc5471-d12c-4565-b3e4-03c9d450f291 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.728669 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:16.758740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1386/2783] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:16.770062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.771468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:16.790229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.790896 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.791175 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.792583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.793317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.795038 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.804821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1398/2784] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:05:16.809634 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.819160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe5d0-278d-4399-9fec-32de9db59c00 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.825117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.835527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.835527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.835527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.835527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.836059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3235-b8a2-489f-8ba5-cb06a714b407 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.836166 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.836266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d15be-ab31-4e35-94c4-115045f93e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.836266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d15be-ab31-4e35-94c4-115045f93e15 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.836266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d15be-ab31-4e35-94c4-115045f93e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.836266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d15be-ab31-4e35-94c4-115045f93e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.839374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1399/2785] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:05:16.843389 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:16.844748 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d15be-ab31-4e35-94c4-115045f93e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.846385 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d15be-ab31-4e35-94c4-115045f93e15 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.855908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ebad2-e7da-41d6-9139-ed163141e86f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:16.856530 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1387/2786] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:05:16.860808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:16.861694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.861978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.862286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] PATH_INFO: `/v3/projects/39e64c6b8dca4adb883218f852485e44/users/c3f403874b9d4977927248963436ce0e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:16.878117 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e64c6b8dca4adb883218f852485e44, user_id=c3f403874b9d4977927248963436ce0e, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:16.880881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:16.962462 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1400/2787] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:16 2022] PUT /identity/v3/projects/39e64c6b8dca4adb883218f852485e44/users/c3f403874b9d4977927248963436ce0e/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:16.975874 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc131e14-5e50-4bba-9582-d21208ab8e69 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:16.976170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc131e14-5e50-4bba-9582-d21208ab8e69 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:16.976451 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc131e14-5e50-4bba-9582-d21208ab8e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.004602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-bc131e14-5e50-4bba-9582-d21208ab8e69 None None] MFA Rules not processed for user `c3f403874b9d4977927248963436ce0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:17.005810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bc131e14-5e50-4bba-9582-d21208ab8e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:17.069617 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1388/2788] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:17.074118 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:17.134241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863f313-340b-48ba-95d4-2921e27baecf admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:17.139344 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:17.140007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.140321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.140599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.141288 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:17.143407 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:17.264418 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1401/2789] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:17 2022] POST /identity/v3/projects => generated 367 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:17.265933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc131e14-5e50-4bba-9582-d21208ab8e69 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:17.271220 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:17.277953 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc131e14-5e50-4bba-9582-d21208ab8e69 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:17.333480 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:17.334160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.334468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.334747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.335434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:17.337376 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:17.483280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1389/2790] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:17 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:17.487256 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:17.497699 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21657b45-0b39-4dd8-9214-d8e451a7495a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:17.502359 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:17.503021 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.503311 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.503588 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.504318 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:17.505939 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:17.531177 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1402/2791] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:17.534291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:17.544497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb1978-75e0-43a2-90ca-cbadeb58da04 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:17.549053 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:17.549943 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.550247 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.550527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] PATH_INFO: `/v3/projects/952e0e4d6f5e4566b0c3e29f2e4ed5f4/users/18a9af661b344154b489f236f0b21439/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.566722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952e0e4d6f5e4566b0c3e29f2e4ed5f4, user_id=18a9af661b344154b489f236f0b21439, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:17.569390 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:17.647498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1390/2792] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:17 2022] PUT /identity/v3/projects/952e0e4d6f5e4566b0c3e29f2e4ed5f4/users/18a9af661b344154b489f236f0b21439/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:17.648833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:17.660537 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6433d-7ccd-4d98-ba66-d03b31ffc025 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:17.737710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d5da6-5007-4141-accb-16701607ad0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:17.737710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5da6-5007-4141-accb-16701607ad0c admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.737710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5da6-5007-4141-accb-16701607ad0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.737710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5da6-5007-4141-accb-16701607ad0c admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.738393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d5da6-5007-4141-accb-16701607ad0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:17.738611 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d5da6-5007-4141-accb-16701607ad0c admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:17.753994 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1403/2793] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:17.755189 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:17.767613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a75853-459d-4211-aee6-0e6e79569fc5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:17.774353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:17.775398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.775571 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.775933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] PATH_INFO: `/v3/projects/952e0e4d6f5e4566b0c3e29f2e4ed5f4/users/18a9af661b344154b489f236f0b21439/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.778263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=952e0e4d6f5e4566b0c3e29f2e4ed5f4, user_id=18a9af661b344154b489f236f0b21439, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:17.787130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:17.801214 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1391/2794] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:17 2022] PUT /identity/v3/projects/952e0e4d6f5e4566b0c3e29f2e4ed5f4/users/18a9af661b344154b489f236f0b21439/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:17.802998 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707de3df-fdc0-4204-ba05-d46cfd13a105 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.803298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707de3df-fdc0-4204-ba05-d46cfd13a105 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.803577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707de3df-fdc0-4204-ba05-d46cfd13a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:17.832572 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-707de3df-fdc0-4204-ba05-d46cfd13a105 None None] MFA Rules not processed for user `18a9af661b344154b489f236f0b21439`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:17.833801 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-707de3df-fdc0-4204-ba05-d46cfd13a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:17.905583 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1404/2795] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:17.921936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:17.986057 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814bf1d-14b7-40df-a5b6-36f7a44a43ff admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:17.989793 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:17.990417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:17.990703 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:17.990981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:18.000864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:18.002394 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:18.012678 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1392/2796] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:18.162322 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:18.215393 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707de3df-fdc0-4204-ba05-d46cfd13a105 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:18.225306 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707de3df-fdc0-4204-ba05-d46cfd13a105 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:18.229474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1012ac9-99f1-45ee-a9af-b21a41197f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:18.239531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1012ac9-99f1-45ee-a9af-b21a41197f1d service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:18.239531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1012ac9-99f1-45ee-a9af-b21a41197f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:18.239531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1012ac9-99f1-45ee-a9af-b21a41197f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:18.287192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1012ac9-99f1-45ee-a9af-b21a41197f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:18.289205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1012ac9-99f1-45ee-a9af-b21a41197f1d service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:18.299721 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1405/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:18.774580 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:20.368365 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:21.247075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:21.310439 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dd341-24d6-4f38-b9b0-40c2eb4b6895 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:21.314900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:21.315886 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:21.316226 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:21.316449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] PATH_INFO: `/v3/users/a2d7054acaad49c2beb0bbd0ba5f9e0a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:21.317613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d7054acaad49c2beb0bbd0ba5f9e0a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:21.319564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:21.588553 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d7054acaad49c2beb0bbd0ba5f9e0a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:21.592092 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d7054acaad49c2beb0bbd0ba5f9e0a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:21.592891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d7054acaad49c2beb0bbd0ba5f9e0a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:21.606187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d7054acaad49c2beb0bbd0ba5f9e0a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:21.732483 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d7054acaad49c2beb0bbd0ba5f9e0a'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:21.745577 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1393/2798] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:21 2022] DELETE /identity/v3/users/a2d7054acaad49c2beb0bbd0ba5f9e0a => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:21.750092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:21.750617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:21.750727 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:21.768648 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:21.769844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:21.833182 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1406/2799] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:21.843246 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:21.845011 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:21.904399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e757a-b088-4360-a743-18dd9392af4b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:21.907961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:21.908601 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:21.908882 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:21.909153 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:21.918914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:21.920366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:21.929694 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1394/2800] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:21.979635 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:22.467097 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:22.480489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:22.484006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac207f7-6c3a-4643-9b28-41acce828a3f None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:22.491374 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:22.492966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:22.493362 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:22.493775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] PATH_INFO: `/v3/projects/688205eadcc14900913fa218d1bb2889` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:22.495953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688205eadcc14900913fa218d1bb2889)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:22.498991 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:22.689793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688205eadcc14900913fa218d1bb2889', 'request_id': 'req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:22.696131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688205eadcc14900913fa218d1bb2889', 'request_id': 'req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:22.813295 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688205eadcc14900913fa218d1bb2889', 'request_id': 'req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:22.829335 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] The token cache is being invalidate because project 688205eadcc14900913fa218d1bb2889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:22.829632 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:22.831293 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-2a4462aa-11ff-4dc6-aade-05a8d4a940f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:22.833160 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1407/2801] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:22 2022] DELETE /identity/v3/projects/688205eadcc14900913fa218d1bb2889 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:23.406961 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:24.413560 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:24.728119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:24.905440 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:25.178400 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:25.312880 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:25.314304 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:25.327062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f138a88-ef21-41b7-bc2a-299d0b6f3acb service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:25.382851 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:25.383731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:25.384028 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:25.384306 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] PATH_INFO: `/v3/users/c3f403874b9d4977927248963436ce0e` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:25.385564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f403874b9d4977927248963436ce0e)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:25.387534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:25.522248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:25.632944 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f403874b9d4977927248963436ce0e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:25.633602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f403874b9d4977927248963436ce0e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:25.634224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f403874b9d4977927248963436ce0e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:25.646742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f403874b9d4977927248963436ce0e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:25.824986 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f403874b9d4977927248963436ce0e'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:25.838955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1395/2802] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:25 2022] DELETE /identity/v3/users/c3f403874b9d4977927248963436ce0e => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:25.843591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:25.844073 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:25.844619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:25.863814 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:25.865033 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:25.871155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:25.935203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1408/2803] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:25.944838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:25.946272 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:26.005425 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f721f-1890-4c8c-830c-d1d331ac3c92 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:26.010567 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:26.011242 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:26.011528 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:26.011805 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:26.021350 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:26.022846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:26.033312 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1396/2804] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:26.613800 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=100 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:26.712269 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:26.714356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:26.730070 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa1a59-12a3-4f57-a030-62f66f02fbf9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:26.735050 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:26.735916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:26.736195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:26.736467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] PATH_INFO: `/v3/projects/39e64c6b8dca4adb883218f852485e44` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:26.737714 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e64c6b8dca4adb883218f852485e44)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:26.740516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:26.852713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:26.911507 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e64c6b8dca4adb883218f852485e44', 'request_id': 'req-d88784cd-5182-47a3-a380-48e27c7023cb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:26.919464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e64c6b8dca4adb883218f852485e44', 'request_id': 'req-d88784cd-5182-47a3-a380-48e27c7023cb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.019959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:27.053356 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e64c6b8dca4adb883218f852485e44', 'request_id': 'req-d88784cd-5182-47a3-a380-48e27c7023cb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.064828 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] The token cache is being invalidate because project 39e64c6b8dca4adb883218f852485e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:27.065113 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.065728 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.067360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1409/2805] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:26 2022] DELETE /identity/v3/projects/39e64c6b8dca4adb883218f852485e44 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:27.071501 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:27.081891 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e15b0-aa66-4142-8f15-0086a225f3e3 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:27.136876 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:27.137796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:27.138102 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:27.138399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] PATH_INFO: `/v3/users/18a9af661b344154b489f236f0b21439` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:27.139588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a9af661b344154b489f236f0b21439)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:27.141662 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:27.333479 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:27.460468 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a9af661b344154b489f236f0b21439'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.461316 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a9af661b344154b489f236f0b21439'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.462033 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a9af661b344154b489f236f0b21439'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.475902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a9af661b344154b489f236f0b21439'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.579622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-5903b66b-d10a-4d55-8d4b-d69cf80062b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a9af661b344154b489f236f0b21439'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:27.593722 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1397/2806] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:27 2022] DELETE /identity/v3/users/18a9af661b344154b489f236f0b21439 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:27.608567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:27.705705 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:27.973676 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:28.095383 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:28.097214 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:28.112703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88784cd-5182-47a3-a380-48e27c7023cb admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:28.173361 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:28.174380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:28.174668 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:28.174963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] PATH_INFO: `/v3/projects/952e0e4d6f5e4566b0c3e29f2e4ed5f4` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:28.176122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=952e0e4d6f5e4566b0c3e29f2e4ed5f4)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:28.178031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:28.225467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:28.366869 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952e0e4d6f5e4566b0c3e29f2e4ed5f4', 'request_id': 'req-f38cd875-f915-42cc-9c92-633fda5190fb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:28.375302 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952e0e4d6f5e4566b0c3e29f2e4ed5f4', 'request_id': 'req-f38cd875-f915-42cc-9c92-633fda5190fb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:28.470669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952e0e4d6f5e4566b0c3e29f2e4ed5f4', 'request_id': 'req-f38cd875-f915-42cc-9c92-633fda5190fb'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:28.481516 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] The token cache is being invalidate because project 952e0e4d6f5e4566b0c3e29f2e4ed5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:28.481805 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:28.482464 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f38cd875-f915-42cc-9c92-633fda5190fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:28.484239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1410/2807] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:28 2022] DELETE /identity/v3/projects/952e0e4d6f5e4566b0c3e29f2e4ed5f4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:28.489208 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1af09-e861-4776-96af-3ac71926069d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:28.489461 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1af09-e861-4776-96af-3ac71926069d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:28.489731 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1af09-e861-4776-96af-3ac71926069d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:28.493426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:28.508359 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-88e1af09-e861-4776-96af-3ac71926069d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:28.510387 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-88e1af09-e861-4776-96af-3ac71926069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:28.573538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e06e-a02a-4f4e-84cb-f6373dd4ae0b None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:28.576653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e06e-a02a-4f4e-84cb-f6373dd4ae0b None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:28.576653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3e06e-a02a-4f4e-84cb-f6373dd4ae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:28.576890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1398/2808] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:28.586975 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193a00e-c7ef-445e-9e05-ea6bf7e5053d None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:28.587356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193a00e-c7ef-445e-9e05-ea6bf7e5053d None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:28.587677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193a00e-c7ef-445e-9e05-ea6bf7e5053d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:28.595832 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-29e3e06e-a02a-4f4e-84cb-f6373dd4ae0b None None] MFA Rules not processed for user `f1314d2b3b8145bbb13aea98702d9b43`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:28.597240 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-29e3e06e-a02a-4f4e-84cb-f6373dd4ae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:28.610297 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9193a00e-c7ef-445e-9e05-ea6bf7e5053d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:28.611583 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9193a00e-c7ef-445e-9e05-ea6bf7e5053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:28.620448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1399/2809] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:28.624446 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193a00e-c7ef-445e-9e05-ea6bf7e5053d None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:28.633209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193a00e-c7ef-445e-9e05-ea6bf7e5053d None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:28.638031 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:28.638776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:28.639062 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:28.639672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:28.640140 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:28.640448 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:28.642046 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:28.654266 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1400/2810] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 15:05:28.658476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:28.669399 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba56f8-df8a-4fd8-b6d6-1aec8c4289ae admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:28.674110 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:28.674879 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:28.675164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:28.675440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:28.676130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:28.678054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:28.722276 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:28.766844 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1411/2811] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:28.818534 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1401/2812] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:28 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:05:28.821011 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3e06e-a02a-4f4e-84cb-f6373dd4ae0b None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:28.836158 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3e06e-a02a-4f4e-84cb-f6373dd4ae0b None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:28.893631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:28.894254 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:28.894543 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:28.894824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:28.895527 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:28.897459 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:28.932680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:28.980511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1412/2813] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:28 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:05:28.984555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:28.991651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:28.998658 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e66f8e-5b3c-4f5c-a02d-511f6c815653 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:29.002036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:29.002747 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:29.003029 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:29.003303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:29.004038 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:29.005754 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:29.016768 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1402/2814] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:29.020853 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:29.033400 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f959a-bd6f-49c6-a066-8604614bb8a7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:29.039111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf6059-8f9b-4e26-89c3-6074e28e5f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:29.039766 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6059-8f9b-4e26-89c3-6074e28e5f33 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:29.040046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6059-8f9b-4e26-89c3-6074e28e5f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:29.040323 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6059-8f9b-4e26-89c3-6074e28e5f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:29.042304 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf6059-8f9b-4e26-89c3-6074e28e5f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:29.043947 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf6059-8f9b-4e26-89c3-6074e28e5f33 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:29.058217 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1413/2815] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:29.059236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:29.069463 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a786f-2996-4652-b952-ebc4145e7b2b admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:29.074917 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:29.075914 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:29.076259 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:29.076625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] PATH_INFO: `/v3/projects/1d3cd9c6d73140eab969f86c929c83d6/users/cbb791e5bfb44889baf0a8128e4996c0/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:29.094704 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3cd9c6d73140eab969f86c929c83d6, user_id=cbb791e5bfb44889baf0a8128e4996c0, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:29.097449 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:29.215684 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1403/2816] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:29 2022] PUT /identity/v3/projects/1d3cd9c6d73140eab969f86c929c83d6/users/cbb791e5bfb44889baf0a8128e4996c0/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:29.221337 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4fc81-5b98-4ff3-bef5-24b7f8ee3bfb None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:29.221630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4fc81-5b98-4ff3-bef5-24b7f8ee3bfb None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:29.221915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4fc81-5b98-4ff3-bef5-24b7f8ee3bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:29.241044 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-95d4fc81-5b98-4ff3-bef5-24b7f8ee3bfb None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:29.246026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-95d4fc81-5b98-4ff3-bef5-24b7f8ee3bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:29.302419 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1414/2817] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:05:29.312075 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:29.313837 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:29.370673 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ece80-0926-41ba-8256-e85ef810ced5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:29.374326 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:29.374970 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:29.375256 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:29.375541 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:29.379617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:29.385241 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:29.386736 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:29.401803 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1404/2818] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:29.511659 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:30.642111 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:30.848261 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:30.928916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c2f5-ee05-4c3f-8f4b-3d45b649d46c None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:30.932231 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c2f5-ee05-4c3f-8f4b-3d45b649d46c None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:30.932770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c2f5-ee05-4c3f-8f4b-3d45b649d46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:30.959920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a553c2f5-ee05-4c3f-8f4b-3d45b649d46c None None] MFA Rules not processed for user `cbb791e5bfb44889baf0a8128e4996c0`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:30.961169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a553c2f5-ee05-4c3f-8f4b-3d45b649d46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:31.027257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1415/2819] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:31.031742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:31.041701 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b6f0c-fed6-48d3-ab8a-d797c3863910 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:31.046783 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:31.047429 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:31.047712 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:31.047988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:31.048692 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:31.050676 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:31.191555 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1405/2820] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:31 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:31.195461 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c2f5-ee05-4c3f-8f4b-3d45b649d46c None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:31.206824 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c2f5-ee05-4c3f-8f4b-3d45b649d46c None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:31.263283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:31.263966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:31.264249 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:31.264526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:31.265236 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:31.267207 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:31.379194 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1416/2821] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:31 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:31.383594 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:31.394517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79463b8e-2abb-451d-8b5a-75ce99be1dd0 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:31.399066 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:31.399759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:31.400041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:31.400314 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:31.401084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:31.402742 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:31.413045 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1406/2822] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:31.417340 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:31.427317 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917af177-1859-405d-9b51-3f2bb6fc2e90 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:31.431925 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:31.433492 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:31.433777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:31.434052 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] PATH_INFO: `/v3/projects/d964413f5cf84d8996f5680dae7c2332/users/6080910bf5ef482485253430bfebac74/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:31.449248 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d964413f5cf84d8996f5680dae7c2332, user_id=6080910bf5ef482485253430bfebac74, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:31.451982 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:31.532148 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1417/2823] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:31 2022] PUT /identity/v3/projects/d964413f5cf84d8996f5680dae7c2332/users/6080910bf5ef482485253430bfebac74/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:31.536073 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:31.547282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b1839-4737-4d13-9bab-f4ecd8f16921 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:31.613864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e347c1b-09e2-48fe-8ec3-3abee5cb03c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:31.621921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347c1b-09e2-48fe-8ec3-3abee5cb03c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:31.621921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347c1b-09e2-48fe-8ec3-3abee5cb03c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:31.621921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347c1b-09e2-48fe-8ec3-3abee5cb03c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:31.621921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e347c1b-09e2-48fe-8ec3-3abee5cb03c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:31.621921 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e347c1b-09e2-48fe-8ec3-3abee5cb03c9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:31.639630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1407/2824] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:31.643808 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:31.654538 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd434c-ae7e-4fea-9c22-9a27a7a2354a admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:31.659210 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:31.660108 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:31.660713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:31.660713 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] PATH_INFO: `/v3/projects/d964413f5cf84d8996f5680dae7c2332/users/6080910bf5ef482485253430bfebac74/roles/2ffcf751c1e14a28ad9d1962a027c04b` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:31.662802 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d964413f5cf84d8996f5680dae7c2332, user_id=6080910bf5ef482485253430bfebac74, role_id=2ffcf751c1e14a28ad9d1962a027c04b)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:31.665431 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:31.678004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1418/2825] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:31 2022] PUT /identity/v3/projects/d964413f5cf84d8996f5680dae7c2332/users/6080910bf5ef482485253430bfebac74/roles/2ffcf751c1e14a28ad9d1962a027c04b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 15:05:31.939515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:32.002580 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:33.180291 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:33.180912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:33.180912 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:33.209978 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] MFA Rules not processed for user `6080910bf5ef482485253430bfebac74`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:33.211320 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:33.294870 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1408/2826] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:33.311466 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:33.314000 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:33.324053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:33.382936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9056aa-5def-43a0-b133-70e77d2d78b7 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:33.386909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:33.387568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:33.387941 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:33.388161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:33.390523 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:33.448161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:33.449738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:33.459694 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1419/2827] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:34.602477 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:34.984864 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:35.076430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:35.483336 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:35.484729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:35.498677 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89376ce-11f7-413c-a78e-4046e8cf771f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:35.502301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:35.502954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:35.503244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:35.503560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:35.512645 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:35.514091 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:35.523605 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1409/2828] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:35.853286 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:35.881858 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:36.168844 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:36.471017 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:36.654412 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:36.956942 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:37.271260 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:37.522466 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:39.873851 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:39.875430 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:39.933738 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32fc88-e354-4342-9d82-5ccbd8ed5a94 service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:39.937603 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:39.938334 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:39.938615 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:39.938891 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:39.940775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:40.016398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:40.017821 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:40.029487 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1420/2829] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:05:42.380007 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:42.390330 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfa908-49f1-442a-8a1b-b6c94602b985 service cinder] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:42.393919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:42.394720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:42.394999 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:42.395512 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:42.397280 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:42.411554 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:42.413010 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:42.424202 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1410/2830] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:50.023826 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=70 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:52.419262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:52.533131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:52.602901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4361ad-cfd6-4901-ad5f-ab35c9c7ce87 service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:52.606547 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:52.607735 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:52.608030 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:52.608315 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] PATH_INFO: `/v3/users/cbb791e5bfb44889baf0a8128e4996c0` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:52.609533 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb791e5bfb44889baf0a8128e4996c0)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:52.611848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:52.884053 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb791e5bfb44889baf0a8128e4996c0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:52.884664 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb791e5bfb44889baf0a8128e4996c0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:52.892411 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb791e5bfb44889baf0a8128e4996c0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:52.999959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb791e5bfb44889baf0a8128e4996c0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:53.012426 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb791e5bfb44889baf0a8128e4996c0'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:53.019881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1421/2831] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:52 2022] DELETE /identity/v3/users/cbb791e5bfb44889baf0a8128e4996c0 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:53.316801 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:53.327799 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e2efb-b917-4519-983f-8691b7874234 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:53.385901 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:53.386810 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:53.387100 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:53.387381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] PATH_INFO: `/v3/projects/1d3cd9c6d73140eab969f86c929c83d6` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:53.388614 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3cd9c6d73140eab969f86c929c83d6)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:53.391471 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:53.514895 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3cd9c6d73140eab969f86c929c83d6', 'request_id': 'req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:53.624478 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3cd9c6d73140eab969f86c929c83d6', 'request_id': 'req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:53.630900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3cd9c6d73140eab969f86c929c83d6', 'request_id': 'req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:53.643609 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] The token cache is being invalidate because project 1d3cd9c6d73140eab969f86c929c83d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:53.643900 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:53.644550 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:53.646138 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1411/2832] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:53 2022] DELETE /identity/v3/projects/1d3cd9c6d73140eab969f86c929c83d6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:53.650139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:53.651620 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:53.665445 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87297f-b61d-4bb6-b28d-75946ddc542b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:53.729606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:53.730563 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:53.730848 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:53.731261 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] PATH_INFO: `/v3/users/6080910bf5ef482485253430bfebac74` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:53.733168 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6080910bf5ef482485253430bfebac74)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:53.735140 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:53.757489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:54.042512 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6080910bf5ef482485253430bfebac74'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.043262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6080910bf5ef482485253430bfebac74'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.050061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6080910bf5ef482485253430bfebac74'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.207138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6080910bf5ef482485253430bfebac74'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.220579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-227f704c-1a33-43a1-b378-948562cc41d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6080910bf5ef482485253430bfebac74'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.226568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1422/2833] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:05:53 2022] DELETE /identity/v3/users/6080910bf5ef482485253430bfebac74 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:54.476995 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:54.478481 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:54.491915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005a266-9f2a-4ef5-a3a7-0d27990af8b4 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:54.550497 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:54.551373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:54.551656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:54.551955 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] PATH_INFO: `/v3/projects/d964413f5cf84d8996f5680dae7c2332` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:54.553148 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d964413f5cf84d8996f5680dae7c2332)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:54.555130 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:54.698506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:54.710855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd964413f5cf84d8996f5680dae7c2332', 'request_id': 'req-ec5814dd-a329-490f-a7ae-3b10e4bd4455'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.834467 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd964413f5cf84d8996f5680dae7c2332', 'request_id': 'req-ec5814dd-a329-490f-a7ae-3b10e4bd4455'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.840938 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd964413f5cf84d8996f5680dae7c2332', 'request_id': 'req-ec5814dd-a329-490f-a7ae-3b10e4bd4455'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.852796 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] The token cache is being invalidate because project d964413f5cf84d8996f5680dae7c2332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:05:54.853198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.853835 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-ec5814dd-a329-490f-a7ae-3b10e4bd4455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:05:54.855410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1412/2834] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:05:54 2022] DELETE /identity/v3/projects/d964413f5cf84d8996f5680dae7c2332 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:54.860810 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281df6d2-8c24-48cf-93c0-4ac22932147e None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:54.861321 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281df6d2-8c24-48cf-93c0-4ac22932147e None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:54.861444 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281df6d2-8c24-48cf-93c0-4ac22932147e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:54.879977 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-281df6d2-8c24-48cf-93c0-4ac22932147e None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:54.881169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-281df6d2-8c24-48cf-93c0-4ac22932147e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:54.939282 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1423/2835] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:54.948135 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77648-bcc5-435a-8e48-e61b5484c6ac None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:54.948427 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77648-bcc5-435a-8e48-e61b5484c6ac None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:54.948734 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77648-bcc5-435a-8e48-e61b5484c6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:54.968405 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-b1a77648-bcc5-435a-8e48-e61b5484c6ac None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:54.969866 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-b1a77648-bcc5-435a-8e48-e61b5484c6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:54.978236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1413/2836] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:54.983112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281df6d2-8c24-48cf-93c0-4ac22932147e None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:54.992811 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281df6d2-8c24-48cf-93c0-4ac22932147e None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:54.998057 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:54.998775 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:54.999059 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:54.999339 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.000088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87001) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:05:55.000399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:55.002018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:55.013796 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1424/2837] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:05:55.018036 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a77648-bcc5-435a-8e48-e61b5484c6ac None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:55.027441 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a77648-bcc5-435a-8e48-e61b5484c6ac None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:55.032353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:55.033065 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:55.033348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:55.033622 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.034389 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:55.036373 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:55.134672 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1414/2838] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:05:55 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:55.138815 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:55.148914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6043681-a1d1-4195-b51a-378c0c2b4f2b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:55.206408 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:55.207004 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:55.207289 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:55.207568 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] PATH_INFO: `/v3/users` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.208246 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:55.210195 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:55.299061 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1425/2839] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:05:55 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:05:55.303380 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:55.313928 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10126686-f6c8-4e0a-8e68-cb3e93a3109d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:55.318551 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:55.319216 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:55.319484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:55.319755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.320476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:55.322084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:55.333178 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1415/2840] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 15:05:55.337257 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:55.348310 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25f179-5b71-4efe-b7be-1397f65fa2ca admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:55.352881 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd175fe2-7ce9-4773-b80e-11cc6f922ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:55.353529 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd175fe2-7ce9-4773-b80e-11cc6f922ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:55.353812 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd175fe2-7ce9-4773-b80e-11cc6f922ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:55.354092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd175fe2-7ce9-4773-b80e-11cc6f922ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.354842 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd175fe2-7ce9-4773-b80e-11cc6f922ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:55.357607 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd175fe2-7ce9-4773-b80e-11cc6f922ac3 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:55.367369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1426/2841] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:05:55.371454 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:55.381420 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478f771-fba9-4fa4-af04-c24bb7438a0e admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:55.386099 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:55.387005 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] REQUEST_METHOD: `PUT` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:55.387331 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:55.387647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] PATH_INFO: `/v3/projects/0ea5456dab0a410680df4ba1a2ae7c70/users/6bceb55f5f454b3f9645ad48de9ac3bd/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.401871 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea5456dab0a410680df4ba1a2ae7c70, user_id=6bceb55f5f454b3f9645ad48de9ac3bd, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:55.405227 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:55.453041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1416/2842] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:05:55 2022] PUT /identity/v3/projects/0ea5456dab0a410680df4ba1a2ae7c70/users/6bceb55f5f454b3f9645ad48de9ac3bd/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:05:55.468655 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde01ef-0e5b-4735-8dd8-2a2e909b3561 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:55.468877 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde01ef-0e5b-4735-8dd8-2a2e909b3561 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:55.469155 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde01ef-0e5b-4735-8dd8-2a2e909b3561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.476102 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:55.495319 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-1dde01ef-0e5b-4735-8dd8-2a2e909b3561 None None] MFA Rules not processed for user `6bceb55f5f454b3f9645ad48de9ac3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:55.496518 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-1dde01ef-0e5b-4735-8dd8-2a2e909b3561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:55.561658 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1427/2843] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:05:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:55.572770 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:55.574517 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:55.635808 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7219a0-3844-4ba5-adbc-ef08b1a2aacb admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:55.641368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:55.641368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:55.641368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:55.641368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:55.650002 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:55.651522 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:55.661009 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1417/2844] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:05:56.769553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:56.770041 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:56.770146 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:56.788474 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:05:56.789679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:56.847866 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1428/2845] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Jun 15 15:05:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:05:56.887219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:05:56.888756 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:05:56.959470 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:05:56.966759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaa107-a6a8-4301-8889-1686afceac8a service glance] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:05:56.972327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:05:56.972976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:05:56.973262 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:05:56.973632 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:05:56.982649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:05:56.984324 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:05:56.992952 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1418/2846] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:05:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:06:06.839978 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=55 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:07.087581 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=5 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:14.776138 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:14.779521 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:14.925969 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445c8c-de13-4fd0-ad5c-a9c469c7c1a9 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:14.929840 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55f909-3913-4277-aa1e-73c921454d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:14.930550 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f909-3913-4277-aa1e-73c921454d67 service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:14.930837 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f909-3913-4277-aa1e-73c921454d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:14.931119 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f909-3913-4277-aa1e-73c921454d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:14.933055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-3e55f909-3913-4277-aa1e-73c921454d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:14.944278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55f909-3913-4277-aa1e-73c921454d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:14.945736 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55f909-3913-4277-aa1e-73c921454d67 service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:14.956024 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1429/2847] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:20.510602 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:20.574410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bb4bc-fb6c-4f25-b7cf-b79ce50ccb49 service swift] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:20.578904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:20.579782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:20.580069 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:20.580346 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] PATH_INFO: `/v3/users/6bceb55f5f454b3f9645ad48de9ac3bd` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:20.581548 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bceb55f5f454b3f9645ad48de9ac3bd)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:20.583537 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:20.844402 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bceb55f5f454b3f9645ad48de9ac3bd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:20.845209 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bceb55f5f454b3f9645ad48de9ac3bd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:20.845823 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bceb55f5f454b3f9645ad48de9ac3bd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:20.861263 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bceb55f5f454b3f9645ad48de9ac3bd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:20.955562 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bceb55f5f454b3f9645ad48de9ac3bd'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:20.970087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1419/2848] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:06:20 2022] DELETE /identity/v3/users/6bceb55f5f454b3f9645ad48de9ac3bd => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:20.974960 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b799f-e384-4e46-8487-0425d5bcb43d None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:20.975504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b799f-e384-4e46-8487-0425d5bcb43d None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:20.975606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b799f-e384-4e46-8487-0425d5bcb43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:20.993666 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-170b799f-e384-4e46-8487-0425d5bcb43d None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:20.994900 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-170b799f-e384-4e46-8487-0425d5bcb43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:21.055222 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1430/2849] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:06:21.065841 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:21.067264 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:21.127170 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661569-057b-4882-a66d-42164c071d0a admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:21.130702 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:21.131347 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:21.131630 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:21.131907 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:21.143638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:21.149303 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:21.156960 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1420/2850] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:21.448213 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:21.448504 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:21.448856 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:21.467484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:21.468708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:21.531031 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1431/2851] 10.222.0.23 () {58 vars in 967 bytes} [Wed Jun 15 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:21.541211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:21.542858 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:21.570743 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:21.617238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781e803-ce79-4d0a-818f-2ea74f0dd090 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:21.625270 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68387941-ce01-45d1-b289-eff9cd2f5f60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:21.626224 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68387941-ce01-45d1-b289-eff9cd2f5f60 service swift] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:21.626649 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68387941-ce01-45d1-b289-eff9cd2f5f60 service swift] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:21.627052 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68387941-ce01-45d1-b289-eff9cd2f5f60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:21.642254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68387941-ce01-45d1-b289-eff9cd2f5f60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:21.642254 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68387941-ce01-45d1-b289-eff9cd2f5f60 service swift] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:21.649765 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1421/2852] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:06:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 15:06:21.671206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:21.671206 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:21.690203 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35792b73-3952-4859-aabb-6db2f3adc397 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:21.695403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:21.696414 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:21.696702 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:21.697105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] PATH_INFO: `/v3/projects/0ea5456dab0a410680df4ba1a2ae7c70` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:21.698298 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea5456dab0a410680df4ba1a2ae7c70)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:21.700934 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:21.850392 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:21.853919 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea5456dab0a410680df4ba1a2ae7c70', 'request_id': 'req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:21.860653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea5456dab0a410680df4ba1a2ae7c70', 'request_id': 'req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:21.950192 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea5456dab0a410680df4ba1a2ae7c70', 'request_id': 'req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:21.961981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] The token cache is being invalidate because project 0ea5456dab0a410680df4ba1a2ae7c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:06:21.962299 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:21.962967 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-e7992a0a-3a63-47be-98c3-e0ddd53ebb9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:21.964640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1432/2853] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:06:21 2022] DELETE /identity/v3/projects/0ea5456dab0a410680df4ba1a2ae7c70 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:21.970085 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe630afc-e8b7-492d-8cf6-82a732a0819f None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:21.970419 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe630afc-e8b7-492d-8cf6-82a732a0819f None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:21.970750 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe630afc-e8b7-492d-8cf6-82a732a0819f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:21.988956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-fe630afc-e8b7-492d-8cf6-82a732a0819f None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:21.990155 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-fe630afc-e8b7-492d-8cf6-82a732a0819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:22.050842 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1422/2854] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:22.058962 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55371cfe-ccea-490e-b536-2f77a19f92ac None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.063016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55371cfe-ccea-490e-b536-2f77a19f92ac None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.063016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55371cfe-ccea-490e-b536-2f77a19f92ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.078448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-55371cfe-ccea-490e-b536-2f77a19f92ac None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:22.079755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-55371cfe-ccea-490e-b536-2f77a19f92ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:22.087726 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1433/2855] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:22.106363 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe630afc-e8b7-492d-8cf6-82a732a0819f None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:22.116229 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe630afc-e8b7-492d-8cf6-82a732a0819f None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:22.120980 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:22.121806 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.122090 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.122398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.123181 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:06:22.123643 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:22.125211 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:22.138238 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1423/2856] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:06:22.142612 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55371cfe-ccea-490e-b536-2f77a19f92ac None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:22.152722 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55371cfe-ccea-490e-b536-2f77a19f92ac None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:22.157457 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:22.160347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.160347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.160347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] PATH_INFO: `/v3/projects` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.160347 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:22.161296 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:22.242737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1434/2857] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:06:22 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:06:22.247023 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:22.257258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a838b1c-c263-428b-a15d-1ac702b32b14 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:22.315804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:22.316476 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.316759 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.317094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.317771 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:22.319887 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:22.483834 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1424/2858] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:06:22 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 15:06:22.485360 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:22.496987 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d60a0-dab5-42b2-9dff-b340234c7091 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:22.501640 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:22.502448 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.507006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.507006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.507006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:22.507006 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:22.517936 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1435/2859] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:06:22.522097 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:22.535318 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297112d6-d61e-40dd-83ac-f3ae152f90d5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:22.540014 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799c3dd-1020-4d6e-8bef-243f170d98ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:22.540689 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799c3dd-1020-4d6e-8bef-243f170d98ff admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.540976 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799c3dd-1020-4d6e-8bef-243f170d98ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.541252 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799c3dd-1020-4d6e-8bef-243f170d98ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.541987 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799c3dd-1020-4d6e-8bef-243f170d98ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:22.544184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799c3dd-1020-4d6e-8bef-243f170d98ff admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:22.556964 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1425/2860] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:06:22.562124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:22.573241 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ec4b9-0d38-4fc9-866c-97a6a966e9c0 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:22.579116 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15537075-325e-4534-a4da-feb340784763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:22.580012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537075-325e-4534-a4da-feb340784763 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.581540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537075-325e-4534-a4da-feb340784763 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.581540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15537075-325e-4534-a4da-feb340784763 admin admin] PATH_INFO: `/v3/projects/b8dd44c980a341daae89968af6df836a/users/986d113276fd45469c454e38b66ec06a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.597816 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15537075-325e-4534-a4da-feb340784763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8dd44c980a341daae89968af6df836a, user_id=986d113276fd45469c454e38b66ec06a, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:22.601397 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15537075-325e-4534-a4da-feb340784763 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:22.731953 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1436/2861] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:06:22 2022] PUT /identity/v3/projects/b8dd44c980a341daae89968af6df836a/users/986d113276fd45469c454e38b66ec06a/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:22.796210 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.796442 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.796670 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.824640 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] MFA Rules not processed for user `986d113276fd45469c454e38b66ec06a`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:22.825847 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:22.830761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:22.890915 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1426/2862] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:22.902847 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15537075-325e-4534-a4da-feb340784763 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:22.904956 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-15537075-325e-4534-a4da-feb340784763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:22.962910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15537075-325e-4534-a4da-feb340784763 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:22.967348 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:22.968007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:22.968291 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:22.968567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:22.977756 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:22.979228 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:22.989200 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1437/2863] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:24.987282 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:24.989560 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:25.056884 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d155b-a63d-458f-b71d-9fb83a7278c5 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:25.062001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:25.062947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:25.063230 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:25.063503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] PATH_INFO: `/v3/users/f1314d2b3b8145bbb13aea98702d9b43` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:25.064686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1314d2b3b8145bbb13aea98702d9b43)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:25.066698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:25.289484 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1314d2b3b8145bbb13aea98702d9b43'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:25.290319 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1314d2b3b8145bbb13aea98702d9b43'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:25.291086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1314d2b3b8145bbb13aea98702d9b43'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:25.305656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1314d2b3b8145bbb13aea98702d9b43'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:25.412954 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-e5c9114e-d670-4ddf-b16a-859ff18c8462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1314d2b3b8145bbb13aea98702d9b43'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:25.428713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1427/2864] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:06:24 2022] DELETE /identity/v3/users/f1314d2b3b8145bbb13aea98702d9b43 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:25.676762 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:25.687971 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e99d6-8c64-4194-939b-28d1edd1d51e service cinder] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:25.751088 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:25.751966 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:25.752250 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:25.752526 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] PATH_INFO: `/v3/projects/a5c6a78958cb4d34862e51557f13d0b1` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:25.753910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c6a78958cb4d34862e51557f13d0b1)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:25.756013 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:25.773630 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:25.918596 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c6a78958cb4d34862e51557f13d0b1', 'request_id': 'req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:25.924927 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c6a78958cb4d34862e51557f13d0b1', 'request_id': 'req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:26.027196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c6a78958cb4d34862e51557f13d0b1', 'request_id': 'req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:26.041161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] The token cache is being invalidate because project a5c6a78958cb4d34862e51557f13d0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:06:26.041161 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:26.042911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-d87c2e3e-fd28-4b6e-87d1-c173a4c772ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:26.044627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1438/2865] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:06:25 2022] DELETE /identity/v3/projects/a5c6a78958cb4d34862e51557f13d0b1 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:26.049761 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78706215-861c-4401-9de3-a0eb5d3d8c9c None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.050301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78706215-861c-4401-9de3-a0eb5d3d8c9c None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.050506 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78706215-861c-4401-9de3-a0eb5d3d8c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.068773 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-78706215-861c-4401-9de3-a0eb5d3d8c9c None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:26.069997 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-78706215-861c-4401-9de3-a0eb5d3d8c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:26.091588 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:26.130647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1428/2866] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:26.139104 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e22e-b053-473a-9530-56f9e01ff706 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.139401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e22e-b053-473a-9530-56f9e01ff706 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.139679 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e22e-b053-473a-9530-56f9e01ff706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.159438 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-a5d6e22e-b053-473a-9530-56f9e01ff706 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:26.160997 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-a5d6e22e-b053-473a-9530-56f9e01ff706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:26.170262 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1439/2867] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:26.174477 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78706215-861c-4401-9de3-a0eb5d3d8c9c None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.185265 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78706215-861c-4401-9de3-a0eb5d3d8c9c None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.192236 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.193001 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.193379 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.193800 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.194570 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87000) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 15:06:26.196740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.196901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.196901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.196901 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.197146 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.213223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1429/2868] 10.222.0.23 () {58 vars in 1152 bytes} [Wed Jun 15 15:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 15:06:26.214540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] MFA Rules not processed for user `fd1b940712c1481ba53ae1b45c68c347`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:26.214786 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.215963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:26.225016 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7812d0d-7a51-4370-ba3d-d3163b7d92a8 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.230408 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.231087 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.231367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.231647 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.232333 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.234356 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.273242 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1440/2869] 10.222.0.23 () {58 vars in 967 bytes} [Wed Jun 15 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:06:26.285585 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.286916 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:26.320514 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1430/2870] 10.222.0.23 () {60 vars in 1152 bytes} [Wed Jun 15 15:06:26 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:06:26.323041 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.359046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885a7ed-8b91-42a3-98d9-fda9fc0b4d96 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.384890 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce181495-375f-49fa-a337-ccf992db71b6 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.389956 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.390657 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.390946 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.391225 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] PATH_INFO: `/v3/users` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.391919 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.393889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.398975 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:26.419708 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3a2fb853764be199cc6ff0dfd18064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.437536 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.437536 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.437536 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.476055 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.477669 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.486591 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1441/2871] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Jun 15 15:06:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 15:06:26.552436 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1431/2872] 10.222.0.23 () {60 vars in 1143 bytes} [Wed Jun 15 15:06:26 2022] POST /identity/v3/users => generated 285 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 15:06:26.553633 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.565112 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9957d-884a-4a9e-8ac1-f5282d6d45ab service swift] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.569784 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.570463 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.571252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.571252 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.571703 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.573285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.583517 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1442/2873] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:06:26.587472 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.599106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b9b7c-fece-4ce3-be72-f8557bb1349d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.603846 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa654-a519-48f6-9fef-1cf959147330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.604557 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa654-a519-48f6-9fef-1cf959147330 admin admin] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.604885 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa654-a519-48f6-9fef-1cf959147330 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.605920 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa654-a519-48f6-9fef-1cf959147330 admin admin] PATH_INFO: `/v3/roles` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.606434 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa654-a519-48f6-9fef-1cf959147330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.608117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa654-a519-48f6-9fef-1cf959147330 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.618540 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1432/2874] 10.222.0.23 () {58 vars in 1121 bytes} [Wed Jun 15 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 15:06:26.622914 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.632403 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e1a4b-a958-40cc-89dd-03ee780821b2 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.637540 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.638451 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.638734 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.639007 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] PATH_INFO: `/v3/projects/c99e60c32bc146dbbb5676995bee80dd/users/52507294a34f47789e750674f6464829/roles/926cbd7dfa704847bbfbf9d0c2aab5fc` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.655139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99e60c32bc146dbbb5676995bee80dd, user_id=52507294a34f47789e750674f6464829, role_id=926cbd7dfa704847bbfbf9d0c2aab5fc)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:26.657777 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:26.801472 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1443/2875] 10.222.0.23 () {60 vars in 1482 bytes} [Wed Jun 15 15:06:26 2022] PUT /identity/v3/projects/c99e60c32bc146dbbb5676995bee80dd/users/52507294a34f47789e750674f6464829/roles/926cbd7dfa704847bbfbf9d0c2aab5fc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:26.812327 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.812621 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.812904 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.841947 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] MFA Rules not processed for user `52507294a34f47789e750674f6464829`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:26.843186 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:26.910269 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:26.919488 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1433/2876] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:26.927741 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:26.929452 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:26.986131 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01dd89-41d5-4417-9134-7e37715bf5c5 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:26.989760 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28faa0c6ac9b4e3a9528c3199506257c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:26.990434 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:26.990718 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:26.990992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:26.999945 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:27.001396 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:27.010247 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1444/2877] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:06:36.270780 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:36.272278 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:36.341568 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5ecdb-acbb-48f5-b61d-445b392dc681 None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:36.346465 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:36.347348 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:36.347638 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:36.347913 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] PATH_INFO: `/v3/users/833efc71510e4bf8a5418671026a1b29` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:36.349106 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=833efc71510e4bf8a5418671026a1b29)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:36.351086 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:36.580307 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833efc71510e4bf8a5418671026a1b29'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:36.580966 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833efc71510e4bf8a5418671026a1b29'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:36.581582 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833efc71510e4bf8a5418671026a1b29'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:36.593981 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833efc71510e4bf8a5418671026a1b29'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:36.675422 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833efc71510e4bf8a5418671026a1b29'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:36.689503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1434/2878] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:06:36 2022] DELETE /identity/v3/users/833efc71510e4bf8a5418671026a1b29 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:36.957836 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:36.975402 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39310670-9a92-4441-b72c-ed3120962321 service glance] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:37.036255 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:37.052422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:37.052422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:37.052422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:37.052422 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] PATH_INFO: `/v3/projects/019a9c2e2943403ab0700a96af732826` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:37.053126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] RBAC: Authorizing `identity:delete_project(project_id=019a9c2e2943403ab0700a96af732826)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:37.053126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:37.244494 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019a9c2e2943403ab0700a96af732826', 'request_id': 'req-c337a3f1-5527-4416-aea6-80eeeaee327f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.253398 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019a9c2e2943403ab0700a96af732826', 'request_id': 'req-c337a3f1-5527-4416-aea6-80eeeaee327f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.309907 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019a9c2e2943403ab0700a96af732826', 'request_id': 'req-c337a3f1-5527-4416-aea6-80eeeaee327f'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.320905 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] The token cache is being invalidate because project 019a9c2e2943403ab0700a96af732826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:06:37.322511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.322511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.324362 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1445/2879] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:06:36 2022] DELETE /identity/v3/projects/019a9c2e2943403ab0700a96af732826 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:37.328821 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:37.330353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:37.346381 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9dd67-eff9-4010-a314-19213234aa3d admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:37.406713 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:37.407496 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:37.407764 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:37.408032 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] PATH_INFO: `/v3/users/ee0d661b6afe4de3b813e19e1597fcb3` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:37.409198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee0d661b6afe4de3b813e19e1597fcb3)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:37.411184 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:37.436499 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:37.613313 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee0d661b6afe4de3b813e19e1597fcb3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.613899 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee0d661b6afe4de3b813e19e1597fcb3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.614613 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee0d661b6afe4de3b813e19e1597fcb3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.626720 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee0d661b6afe4de3b813e19e1597fcb3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.749143 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee0d661b6afe4de3b813e19e1597fcb3'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:37.762779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1435/2880] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:06:37 2022] DELETE /identity/v3/users/ee0d661b6afe4de3b813e19e1597fcb3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:38.025949 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:38.027909 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:38.049283 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337a3f1-5527-4416-aea6-80eeeaee327f admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:38.111770 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:38.112660 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:38.112951 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:38.113229 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] PATH_INFO: `/v3/projects/66132029bce54dccb5d9a8a4ebe01d99` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:38.114515 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66132029bce54dccb5d9a8a4ebe01d99)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:38.116496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:38.244375 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66132029bce54dccb5d9a8a4ebe01d99', 'request_id': 'req-6b398bd4-7c49-4227-ba34-5eee06bbfb78'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:38.251912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66132029bce54dccb5d9a8a4ebe01d99', 'request_id': 'req-6b398bd4-7c49-4227-ba34-5eee06bbfb78'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:38.360747 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66132029bce54dccb5d9a8a4ebe01d99', 'request_id': 'req-6b398bd4-7c49-4227-ba34-5eee06bbfb78'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:38.373216 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] The token cache is being invalidate because project 66132029bce54dccb5d9a8a4ebe01d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:06:38.373511 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:38.374285 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6b398bd4-7c49-4227-ba34-5eee06bbfb78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:38.376401 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1446/2881] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:06:38 2022] DELETE /identity/v3/projects/66132029bce54dccb5d9a8a4ebe01d99 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:47.758315 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:48.369139 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=79,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:06:58.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:58.975686 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:58.987139 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca7bb9-db30-4b0f-ad55-28c9029307e9 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:59.044782 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:59.045767 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:59.046071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:59.046423 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] PATH_INFO: `/v3/users/52507294a34f47789e750674f6464829` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:59.047620 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52507294a34f47789e750674f6464829)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:59.049656 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:59.441372 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52507294a34f47789e750674f6464829'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:59.442059 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52507294a34f47789e750674f6464829'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:59.442738 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52507294a34f47789e750674f6464829'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:59.454740 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52507294a34f47789e750674f6464829'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:59.554654 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52507294a34f47789e750674f6464829'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:06:59.571410 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1436/2882] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:06:58 2022] DELETE /identity/v3/users/52507294a34f47789e750674f6464829 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:06:59.576071 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] REQUEST_METHOD: `POST` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:59.577012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:59.577012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:59.597121 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.auth.core [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87001) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:06:59.598399 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:59.666653 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1447/2883] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:06:59.677760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:06:59.679223 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:06:59.743729 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289985f-5ef3-4006-a91c-703cc89de2a5 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:06:59.747370 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:06:59.748060 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:06:59.748417 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:06:59.748697 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:06:59.757830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:06:59.759367 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:06:59.769054 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1437/2884] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:00.259092 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:00.260506 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:00.273486 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3c921-6b72-40dc-a65b-192d4c8deb55 None None] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:00.278455 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:00.279458 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:00.282205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:00.282205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] PATH_INFO: `/v3/projects/c99e60c32bc146dbbb5676995bee80dd` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:00.282205 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99e60c32bc146dbbb5676995bee80dd)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:00.283545 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:00.437369 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99e60c32bc146dbbb5676995bee80dd', 'request_id': 'req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:00.443737 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99e60c32bc146dbbb5676995bee80dd', 'request_id': 'req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:00.548963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99e60c32bc146dbbb5676995bee80dd', 'request_id': 'req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:00.560835 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] The token cache is being invalidate because project c99e60c32bc146dbbb5676995bee80dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87001) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:07:00.561122 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:00.561752 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:00.563341 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1448/2885] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:07:00 2022] DELETE /identity/v3/projects/c99e60c32bc146dbbb5676995bee80dd => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:07:03.735084 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:03.736527 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:03.764207 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:03.794164 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16cfbf-87b5-471b-8cc1-06ff4f53cb72 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:03.842219 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98610ef1-dd4f-4f94-bc79-ec00dc6de43d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d29198c768d4024af04273a3cca8f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:03.842949 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98610ef1-dd4f-4f94-bc79-ec00dc6de43d service cinder] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:03.843231 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98610ef1-dd4f-4f94-bc79-ec00dc6de43d service cinder] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:03.843503 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98610ef1-dd4f-4f94-bc79-ec00dc6de43d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:03.845519 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-98610ef1-dd4f-4f94-bc79-ec00dc6de43d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:03.902760 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98610ef1-dd4f-4f94-bc79-ec00dc6de43d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:03.904779 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98610ef1-dd4f-4f94-bc79-ec00dc6de43d service cinder] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:03.914257 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1438/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:10.557016 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:13.104391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:13.105923 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:13.170628 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a17df-65b5-49d9-b27b-d8182b33e7ee admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:13.175476 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:13.176351 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:13.176637 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:13.176912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] PATH_INFO: `/v3/users/986d113276fd45469c454e38b66ec06a` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:13.178084 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=986d113276fd45469c454e38b66ec06a)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:13.180045 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:13.458483 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '986d113276fd45469c454e38b66ec06a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:13.459137 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '986d113276fd45469c454e38b66ec06a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:13.465755 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '986d113276fd45469c454e38b66ec06a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:13.554599 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '986d113276fd45469c454e38b66ec06a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:13.567963 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '986d113276fd45469c454e38b66ec06a'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:13.573910 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1449/2887] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:07:13 2022] DELETE /identity/v3/users/986d113276fd45469c454e38b66ec06a => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:07:13.578498 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50814030-732b-4c08-a729-5706313ff262 None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:13.578967 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50814030-732b-4c08-a729-5706313ff262 None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:13.579076 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50814030-732b-4c08-a729-5706313ff262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:13.595936 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-50814030-732b-4c08-a729-5706313ff262 None None] MFA Rules not processed for user `68464924e2e845d8a0fc9fa4fd5b1824`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:07:13.597154 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-50814030-732b-4c08-a729-5706313ff262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:13.664183 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1439/2888] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 15:07:13.674577 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:13.676010 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:13.748198 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f1d9a-2541-433f-b7b8-433b0a9f1c87 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:13.752484 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:13.752988 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:13.753278 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:13.753558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:13.753962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] REQUEST_METHOD: `POST` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:13.754328 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:13.754564 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:13.766553 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:13.768020 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:13.776248 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.auth.core [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] MFA Rules not processed for user `71f3bb1357f14481bd1850c1153bc253`. Rule list: `[]` (Enabled: `True`). {{(pid=87000) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 15:07:13.777536 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:13.778549 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1450/2889] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:13.814094 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:13.849043 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1440/2890] 10.222.0.23 () {58 vars in 957 bytes} [Wed Jun 15 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 15:07:14.174617 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:14.176012 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:14.187079 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780b870-1cdf-40e2-9224-14332333cdbb service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:14.190937 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:14.192626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:14.192626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:14.192626 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:14.193833 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:14.203791 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:14.205308 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:14.213320 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1451/2891] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:07:14.230349 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:14.231840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:14.246244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0b5b8-55d3-476b-bd39-36e229f460fa None None] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:14.250962 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:14.251833 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:14.252117 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:14.252398 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] PATH_INFO: `/v3/projects/b8dd44c980a341daae89968af6df836a` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:14.253618 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8dd44c980a341daae89968af6df836a)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:14.255804 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:14.382311 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8dd44c980a341daae89968af6df836a', 'request_id': 'req-166d74c3-cb9f-44b1-bdf9-fead5c830e58'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:14.466515 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8dd44c980a341daae89968af6df836a', 'request_id': 'req-166d74c3-cb9f-44b1-bdf9-fead5c830e58'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:14.479646 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8dd44c980a341daae89968af6df836a', 'request_id': 'req-166d74c3-cb9f-44b1-bdf9-fead5c830e58'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:14.485830 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] The token cache is being invalidate because project b8dd44c980a341daae89968af6df836a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:07:14.486119 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:14.487000 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:07:14.488680 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1441/2892] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:07:14 2022] DELETE /identity/v3/projects/b8dd44c980a341daae89968af6df836a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:07:15.029522 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:15.031029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:15.042124 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:15.103992 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56046f3e-439d-4719-a095-8fbe5800952d service neutron] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:15.165959 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:15.166567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:15.166855 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:15.167136 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:15.168912 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:15.183898 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:15.185531 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:15.196920 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1452/2893] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:19.576889 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:19.578902 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:19.671415 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d74c3-cb9f-44b1-bdf9-fead5c830e58 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:19.678337 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:19.680840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:19.680840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:19.680840 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:19.682755 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:19.703991 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:19.705849 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:19.718027 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1442/2894] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:25.320314 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=54 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:29.713508 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:30.945825 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:30.958489 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54efa5-63e7-41e7-b316-d222c06a84b9 service placement] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:30.963911 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac6a657099c14f598d5becd5ab72ee84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:30.964558 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:30.964846 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:30.965126 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:30.966890 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:30.981377 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:30.983101 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:30.994903 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1453/2895] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:34.048019 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:07:34.049795 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:34.108538 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67979b-2a66-41dd-b5e4-5df577b94e3f service nova] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:07:34.113072 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:07:34.113718 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:07:34.113998 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:07:34.114353 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:07:34.116175 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:07:34.178918 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:07:34.180364 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:07:34.190258 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1443/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:07:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:07:40.990280 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=6 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:07:44.185502 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=50 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:09:20.128286 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:20.129793 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:20.221163 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c92c45-0760-49fa-a2a6-f5bf377d64b4 service nova] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:20.227542 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:20.228208 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:20.228496 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:20.228771 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:20.230674 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:20.243798 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:20.245388 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:20.255566 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1454/2897] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:09:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:09:30.251169 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=37 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:09:32.448855 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:32.450300 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:32.503563 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a8d46-7958-410d-b3b6-034c68f0ad74 service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:32.508160 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bead604b09be482c906a97b53bb03d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:32.509244 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:32.509625 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:32.509909 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:32.512440 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:32.600292 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:32.601668 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:32.621071 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1444/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:09:36.187105 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:36.195467 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaac767-a1ca-4c04-b8b0-29dcc27cefe0 service placement] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:36.201417 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:36.202075 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:36.202424 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:36.202720 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:36.204567 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:36.218584 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:36.220391 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:36.232879 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1455/2899] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:09:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:09:40.283776 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:40.296838 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ef2a4-8316-4903-872b-bf4b102ca5cd service neutron] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:40.302445 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:40.303366 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] REQUEST_METHOD: `GET` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:40.303651 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:40.303930 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:40.305869 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:40.318698 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:40.320339 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:40.331165 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1445/2900] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:09:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 15:09:40.467981 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:40.469439 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:40.484606 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52179c42-1868-4268-aca6-2f59b81b222a service placement] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:40.489443 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:40.490350 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:40.490631 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:40.490908 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] PATH_INFO: `/v3/users/71f3bb1357f14481bd1850c1153bc253` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:40.492086 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f3bb1357f14481bd1850c1153bc253)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:40.494018 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:40.748579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f3bb1357f14481bd1850c1153bc253'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:40.749239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f3bb1357f14481bd1850c1153bc253'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:40.757029 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f3bb1357f14481bd1850c1153bc253'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:40.852579 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f3bb1357f14481bd1850c1153bc253'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:40.867272 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f3bb1357f14481bd1850c1153bc253'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:40.875239 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1456/2901] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:09:40 2022] DELETE /identity/v3/users/71f3bb1357f14481bd1850c1153bc253 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:09:41.205213 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:41.218730 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0daaec-a256-4c4c-870c-7b22fad12aff service placement] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:41.290201 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:41.291547 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:41.291979 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:41.292396 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] PATH_INFO: `/v3/projects/547883446fb944e4811768d29eb67583` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:41.294374 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547883446fb944e4811768d29eb67583)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:41.297487 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:41.318933 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=100 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:09:41.455068 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547883446fb944e4811768d29eb67583', 'request_id': 'req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.540187 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547883446fb944e4811768d29eb67583', 'request_id': 'req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.546301 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547883446fb944e4811768d29eb67583', 'request_id': 'req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.558785 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] The token cache is being invalidate because project 547883446fb944e4811768d29eb67583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:09:41.559077 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.559737 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.561288 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1446/2902] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:09:41 2022] DELETE /identity/v3/projects/547883446fb944e4811768d29eb67583 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:09:41.565264 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:41.566710 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:41.580629 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13b5be-9b39-4726-9ba0-de5b6b4b917b admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:41.647371 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:41.648260 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:41.648548 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:41.648827 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] PATH_INFO: `/v3/users/1eff21757485434680444a2c5293a477` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:41.650032 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eff21757485434680444a2c5293a477)` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:41.652026 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:41.937980 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eff21757485434680444a2c5293a477'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.938681 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eff21757485434680444a2c5293a477'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.945984 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eff21757485434680444a2c5293a477'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:41.949924 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:09:42.069346 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eff21757485434680444a2c5293a477'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:42.081636 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.notifications [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eff21757485434680444a2c5293a477'} {{(pid=87001) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:42.088380 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1457/2903] 10.222.0.23 () {58 vars in 1223 bytes} [Wed Jun 15 15:09:41 2022] DELETE /identity/v3/users/1eff21757485434680444a2c5293a477 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:09:42.410513 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Authenticating user token {{(pid=87000) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:09:42.412123 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.fernet_utils [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87000) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:09:42.428963 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fa0bb-5d73-4d16-8109-0889407b6fe9 admin admin] Validating token access rules against request {{(pid=87000) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:09:42.483716 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68464924e2e845d8a0fc9fa4fd5b1824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '108eb4dc74a441578e74c932122b6873', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87000) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:09:42.484610 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:09:42.484988 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:09:42.485371 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] PATH_INFO: `/v3/projects/e050135bf2984c6188050907025aa1ac` {{(pid=87000) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:09:42.486671 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e050135bf2984c6188050907025aa1ac)` {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:09:42.489744 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] RBAC: Authorization granted {{(pid=87000) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:09:42.709710 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e050135bf2984c6188050907025aa1ac', 'request_id': 'req-852c60d2-aaaa-4639-a619-5c81b629df61'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:42.814576 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e050135bf2984c6188050907025aa1ac', 'request_id': 'req-852c60d2-aaaa-4639-a619-5c81b629df61'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:42.821198 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e050135bf2984c6188050907025aa1ac', 'request_id': 'req-852c60d2-aaaa-4639-a619-5c81b629df61'} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:42.832590 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] The token cache is being invalidate because project e050135bf2984c6188050907025aa1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87000) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 15:09:42.832931 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:42.833575 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG keystone.notifications [None req-852c60d2-aaaa-4639-a619-5c81b629df61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87000) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 15:09:42.835158 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: [pid: 87000|app: 0|req: 1447/2904] 10.222.0.23 () {58 vars in 1232 bytes} [Wed Jun 15 15:09:42 2022] DELETE /identity/v3/projects/e050135bf2984c6188050907025aa1ac => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 15:09:52.829368 dv-cinder-845178-7-iscsi devstack@keystone.service[87000]: DEBUG dbcounter [-] [87000] Writing DB stats keystone:SELECT=69,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87000) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:09:53.009046 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 15:15:00.607680 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Authenticating user token {{(pid=87001) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 15:15:00.609578 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:15:00.685437 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2f55-c56c-43c6-ae49-33b080583e04 admin admin] Validating token access rules against request {{(pid=87001) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 15:15:00.747964 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79e4b3-52d9-407b-8e81-f67ec24a0e03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6998e874e8264977a7dc412436162a02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abc7363e773e432ea1d89f126f3dcf5e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87001) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 15:15:00.748627 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79e4b3-52d9-407b-8e81-f67ec24a0e03 service placement] REQUEST_METHOD: `GET` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 15:15:00.748915 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79e4b3-52d9-407b-8e81-f67ec24a0e03 service placement] SCRIPT_NAME: `/identity` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 15:15:00.749196 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79e4b3-52d9-407b-8e81-f67ec24a0e03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87001) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 15:15:00.751040 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.fernet_utils [None req-7d79e4b3-52d9-407b-8e81-f67ec24a0e03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87001) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 15:15:00.763619 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79e4b3-52d9-407b-8e81-f67ec24a0e03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 15:15:00.765225 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79e4b3-52d9-407b-8e81-f67ec24a0e03 service placement] RBAC: Authorization granted {{(pid=87001) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 15:15:00.775611 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: [pid: 87001|app: 0|req: 1458/2905] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Jun 15 15:15:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 15:15:10.771387 dv-cinder-845178-7-iscsi devstack@keystone.service[87001]: DEBUG dbcounter [-] [87001] Writing DB stats keystone:SELECT=58 {{(pid=87001) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}